Universite¶ catholique de Louvain Faculte¶ des Sciences appliquees¶ Departement¶ d'Ingenierie¶ mathematique¶ Center for Operations Research and Econometrics Center for Systems Engineering and Applied Mechanics Spectral Functions and Smoothing Techniques on Jordan Algebras Michel Baes Thesis submitted in partial ful¯llment of the requirements for the degree of Docteur en Sciences Appliqu¶ees Dissertation Committee: Fran»coisGlineur Universit¶ecatholique de Louvain Yurii Nesterov (advisor) Universit¶ecatholique de Louvain Cornelius Roos Technische Universiteit Delft Jean-Pierre Tignol Universit¶ecatholique de Louvain Paul Van Dooren (advisor) Universit¶ecatholique de Louvain Jean-Philippe Vial Universit¶ede Gen`eve Vincent Blondel (public defense chair) Universit¶ecatholique de Louvain Vincent Wertz (private defense chair) Universit¶ecatholique de Louvain ii September 2006 Acknowledgements Although a doctoral dissertation is considered as a personal achievement, it is impossible to complete it without the help of many people. First of all, my gratitude goes to Yurii Nesterov and Paul Van Dooren. I enjoyed the many informal meetings that I had with Yurii Nesterov during the four years of our collaboration. I had the pleasure of bene¯tting from his very deep knowledge of the ¯eld, his amazing intuition, and his never slaked curiosity. Moreover, he gave me the opportunity of meeting a lot of other leading experts in the ¯eld, at CORE and at various conferences. Paul Van Dooren and Yurii Nesterov have been a continuous source of intellectual motivation. Also, I gave them a lot of work in reading sometimes rather obscure previous drafts of my papers and of this thesis. Fran»coisGlineur also helped me a lot in this work. I really enjoyed our frequent and passionate discussions about convex optimization in general while preparing exercises for our students. I would also like to thank Professors Cornelius Roos, Jean-Philippe Vial and Jean-Pierre Tignol for accepting to be members of the jury. They did an excellent work in revising this intolerably long text, and their fruitful comments on its earlier versions helped me to improve it a lot. Many thanks to Vincent Wertz for assuming the charge of president of the jury during my private defense, and to Vincent Blondel for having kindly accepted to chair my public defense. I am thankful to Yvan Hachez for his outstanding patience when he helped me to ¯ll my lengthy FNRS applications. CORE is a fantastic place to work, but also a great place to discover new cultures. I will always remember those numerous international gastronomic dinners organised by Mady, where Bharath, Michal, Marcin, Margaret, Andreas, Malika, Ruslan, and Leysan showed us their secret cooking talents. I have met at CORE wonderful friends from all over the world. A full list of all the people form CORE whose company I had the pleasure to enjoy would be out of reach in this short acknowledgement. However, let me mention Hamish, Quentin, Matthieu, Fran»cois,Peter, Kent, Robert, Jo~ao,Eissa, Helena, Diego, Elina, Alexei, Eloijsa, Ingmar, and Alain for the many discussions and laughs we had together. And, last but not least, Ema for her love and her patience { she even accomplished the great feat of reading this text from cover to cover! iii iv Acknowledgements Finally, I wholeheartedly thank my parents, my grandparents, my sister, and her hus- band for their love and support. Pecunia nervus belli. This thesis has been ¯nancially supported by a FNRS fellow- ship (Belgian Fund for Scienti¯c Research). Moreover, various scienti¯c programs such as IAPV/22 Systems and Control and ARC Large Graphs and Networks provided me funds for attending several scienti¯c meetings. Their support is gratefully acknowledged. Contents Acknowledgements iii List of notation ix 1 Introduction and preliminaries 1 1.1 Comparing algorithms .............................. 2 1.2 Linear Programming ............................... 3 1.3 Convex Programming .............................. 5 1.4 Self-scaled Optimization, and formally real Jordan algebras ......... 10 1.5 A closer look at interior-point methods ..................... 12 1.5.1 Newton's Algorithm: solving unconstrained problems ........ 12 1.5.2 Barrier methods: dealing with constraints ............... 13 1.5.3 Choosing an appropriate barrier .................... 13 1.5.4 Path-following interior-point methods for Linear Programming ... 17 1.5.5 Path-following interior-point methods for Self-Scaled Programming . 18 1.6 Smoothing techniques .............................. 20 1.7 Eigenvalues in Jordan algebra make it work: more applications ....... 22 1.7.1 A concavity result ............................ 22 1.7.2 Augmented barriers in Jordan algebras ................. 23 1.8 Overview of the thesis and research summary ................. 25 2 Jordan algebras 27 2.1 The birth of Jordan algebras .......................... 28 2.2 Algebras and Jordan algebras .......................... 30 2.2.1 Extensions of vector spaces ....................... 30 v vi Contents 2.2.2 Jordan algebras .............................. 32 2.2.3 Strictly power-associative algebras ................... 34 2.2.4 Examples ................................. 36 2.3 Characteristic polynomial ............................ 38 2.3.1 Minimal polynomial over associative and commutative algebras ... 38 2.3.2 Characteristic polynomial over strictly power-associative algebras .. 42 2.3.3 Examples ................................. 54 2.4 Di®erential calculus ............................... 55 2.5 The quadratic operator ............................. 57 2.5.1 De¯nition and ¯rst properties ...................... 58 2.5.2 Quadratic operator and determinant .................. 59 2.5.3 Polarization of the quadratic operator ................. 60 2.5.4 Examples ................................. 61 2.6 Pierce decompositions .............................. 61 2.6.1 An illustrative example ......................... 61 2.6.2 Pierce decomposition theorems and ¯rst consequences ........ 63 2.6.3 Further examples ............................. 67 2.7 Spectral decomposition .............................. 68 2.7.1 Spectral decomposition in power-associative algebras ......... 68 2.7.2 More properties of the determinant ................... 71 2.7.3 Spectral decomposition in formally real Jordan algebras ....... 72 2.7.4 Minimal idempotents ........................... 74 2.7.5 A second spectral decomposition theorem for formally real Jordan algebras .................................. 77 2.7.6 A Euclidean topology in J ....................... 79 2.7.7 Operator commutativity ......................... 80 2.7.8 Eigenvalues of operators ......................... 82 2.7.9 Examples ................................. 84 2.8 Cone of squares .................................. 86 2.8.1 Examples ................................. 89 2.9 Simple Jordan algebras .............................. 89 2.10 Automorphisms .................................. 90 2.10.1 The structure group ........................... 91 2.10.2 Automorphisms of Jordan algebras ................... 93 2.11 Jordan algebras make it work: proofs for Section 1.7 ............. 96 2.11.1 A concavity result ............................ 96 2.11.2 Augmented barriers in Jordan algebras ................. 100 Contents vii 2.12 Conclusion .................................... 102 3 Variational characterizations of eigenvalues 103 3.1 Introduction .................................... 104 3.2 Ky Fan's inequalities ............................... 105 3.3 Subalgebras J1(c) ................................ 109 3.4 Courant-Fischer's Theorem ........................... 112 3.5 Wielandt's Theorem ............................... 118 3.6 Applications of Wielandt's Theorem ...................... 125 4 Spectral functions 129 4.1 Introduction .................................... 130 4.1.1 Functions and di®erentials ........................ 131 4.1.2 Symmetric functions ........................... 133 4.2 Further results on Jordan algebras ....................... 133 4.3 Properties of spectral domains ......................... 138 4.4 Inherited properties of spectral functions .................... 141 4.4.1 The conjugate and the subdi®erential of a spectral function ..... 141 4.4.2 Directional derivative of eigenvalue functions ............. 142 4.4.3 First derivatives of spectral functions .................. 146 4.4.4 Convex properties of spectral functions ................ 149 4.5 Clarke subdi®erentiability ............................ 151 5 Spectral mappings 157 5.1 Introduction .................................... 158 5.2 De¯ning the problem ............................... 159 5.3 Fixing a converging sequence .......................... 160 5.4 Limiting behavior of a sequence of Jordan frames ............... 161 5.5 Jacobian of spectral mapping .......................... 167 5.6 Continuous di®erentiability of spectral mappings ............... 171 5.7 Application: complementarity problems .................... 174 5.7.1 Chen-Mangasarian smoothing functions ................ 176 5.7.2 Fischer-Burmeister smoothing functions ................ 179 6 Smoothing techniques 183 6.1 Introduction .................................... 184 6.2 Smoothing techniques in non-smooth convex optimization .......... 184 6.3 Smoothing for piecewise linear optimization .................. 187 viii Contents 6.4 An upper bound on the Hessian of the power function ............ 189 6.5 Sum-of-norms problem .............................. 194 6.6 Computational experiments