REPORTS IN INFORMATICS ISSN 0333-3590 Sparsity in Higher Order Methods in Optimization Geir Gundersen and Trond Steihaug REPORT NO 327 June 2006 Department of Informatics UNIVERSITY OF BERGEN Bergen, Norway This report has URL http://www.ii.uib.no/publikasjoner/texrap/ps/2006-327.ps Reports in Informatics from Department of Informatics, University of Bergen, Norway, is available at http://www.ii.uib.no/publikasjoner/texrap/. Requests for paper copies of this report can be sent to: Department of Informatics, University of Bergen, Høyteknologisenteret, P.O. Box 7800, N-5020 Bergen, Norway Sparsity in Higher Order Methods in Optimization Geir Gundersen and Trond Steihaug Department of Informatics University of Bergen Norway fgeirg,
[email protected] 29th June 2006 Abstract In this paper it is shown that when the sparsity structure of the problem is utilized higher order methods are competitive to second order methods (Newton), for solving unconstrained optimization problems when the objective function is three times continuously differentiable. It is also shown how to arrange the computations of the higher derivatives. 1 Introduction The use of higher order methods using exact derivatives in unconstrained optimization have not been considered practical from a computational point of view. We will show when the sparsity structure of the problem is utilized higher order methods are competitive compared to second order methods (Newton). The sparsity structure of the tensor is induced by the sparsity structure of the Hessian matrix. This is utilized to make efficient algorithms for Hessian matrices with a skyline structure. We show that classical third order methods, i.e Halley's, Chebyshev's and Super Halley's method, can all be regarded as two step Newton like methods.