Preprint RAL-P-2013-005 On positive semidefinite modification schemes for incomplete Cholesky factorization J Scott, M Tuma April 2013 Submitted for publication in SIAM Journal on Scientific Computing RAL Library STFC Rutherford Appleton Laboratory R61 Harwell Oxford Didcot OX11 0QX Tel: +44(0)1235 445384 Fax: +44(0)1235 446403 email: [email protected] Science and Technology Facilities Council preprints are available online at: http://epubs.stfc.ac.uk ISSN 1361- 4762 Neither the Council nor the Laboratory accept any responsibility for loss or damage arising from the use of information contained in any of their reports or in any communication about their tests or investigations. On positive semidefinite modification schemes for incomplete Cholesky factorization Jennifer Scott1 and Miroslav T˚uma2 ABSTRACT Incomplete Cholesky factorizations have long been important as preconditioners for use in solving large- scale symmetric positive-definite linear systems. In this paper, we present a brief historical overview of the work that has been done over the last fifty years, highlighting key discoveries and rediscoveries. We focus in particular on the relationship between two important positive semidefinite modification schemes, namely that of Jennings and Malik and that of Tismenetsky. We present a novel view of their relationship and implement them in combination with a limited memory approach. We explore their effectiveness using extensive numerical experiments involving a large set of test problems arising from a wide range of practical applications. The experiments are used to isolate the effects of semidefinite modifications to enable their usefulness in the development of robust algebraic incomplete factorization preconditioners to be assessed. We show that we are able to compute sparse incomplete factors that provide robust, general-purpose preconditioners. Keywords: sparse matrices, sparse linear systems, positive-definite symmetric systems, iterative solvers, preconditioning, incomplete Cholesky factorization. AMS(MOS) subject classifications: 65F05, 65F50 1 Computational Science and Engineering Department, Rutherford Appleton Laboratory, Harwell Oxford, Oxfordshire, OX11 0QX, UK. Correspondence to: [email protected] Supported by EPSRC grant EP/I013067/1. 2 Institute of Computer Science, Academy of Sciences of the Czech Republic. Partially supported by the Grant Agency of the Czech Republic Project No. P201/13-06684 S. Travel support from the Academy of Sciences of the Czech Republic is also acknowledged. April 18, 2013 1 Introduction Iterative methods are widely used for the solution of large sparse symmetric linear systems of equations Ax = b. To increase their robustness, the system matrix A generally needs to be transformed by preconditioning. For positive-definite systems, an important class of preconditioners is represented by incomplete Cholesky (IC) factorizations, that is, factorizations of the form LLT in which some of the fill entries (entries that were zero in A) that would occur in a complete factorization are ignored. Over the last fifty years, many different algorithms for computing incomplete factorizations have been proposed and they have been used to solve problems from a wide range of application areas. In Section 2, we provide a brief historical overview and highlight important developments and significant contributions in the field. Some crucial ideas that relate to modifications and dropping are explained in greater detail in Section 3. Here we consider two important semidefinite modification schemes that were introduced to avoid factorization breakdown (that is, zero or negative pivots): that of Jennings and Malik [61, 62] from the mid 1970s and that of Tismenetsky [109] from the early 1990s. Variations of the Jennings-Malik scheme were adopted by engineering communities and it is recommended in, for example, [92] and used in experiments in [11] to solve hard problems, including the analysis of structures and shape optimization. The Tismenetsky approach has also been used to provide a robust preconditioner for some real-world problems, see, for example, [4, 9, 77, 78]. However, in his authoritative survey paper [10], Benzi remarks that Tismenetsky's idea \has unfortunately attracted surprisingly little attention". Benzi also highlights a serious drawback of the scheme which is that its memory requirements can be prohibitively large (in some cases, more than 70 per cent of the storage required for a complete Cholesky factorization is needed, see also [12]). We seek to gain a better understanding of the Jennings-Malik and Tismenetsky semidefinite modifications schemes and to explore the relationship between them. Our main contribution in Section 3 is new theoretical results that compare the 2-norms of the modifications to A that each approach makes. Then, in Section 4, we propose a memory-efficient variant of the Tismenetsky approach, optionally combined with the use of drop tolerances and the Jennings-Malik modifications to reduce factorization breakdowns. In Section 5, we report on extensive numerical experiments in which we aim to isolate the effects of the modifications so as to assess their usefulness in the development of robust algebraic incomplete factorization preconditioners. Finally, we draw some conclusions in Section 6. 2 Historical overview The rapid development of computational tools in the second half of the 20th century significantly pushed forward progress in solving systems of linear algebraic equations. Both software and hardware tools helped to solve still larger and more challenging problems from a wide range of engineering and scientific applications. This further motivated research into the development of improved algorithms and better understanding of their theoretical properties and practical limitations. In particular, within a very short period, important seminal papers were published by Hestenes and Stiefel [51] and Lanczos [74], on which most contemporary iterative solvers are based. The work of Householder and others in the 1950's and early 1960's on factorization approaches to matrix computations substantially changed the field (see [53]). Around the same time (and not long after Turing coined the term preconditioning in [111]), another set of important contributions led to the rapid development of sparse direct methods [93] and preconditioning of iterative methods by incomplete factorizations. In this paper, we consider preconditioning techniques based on incomplete factorizations that belong to the group of algebraically constructed techniques. To better understand contemporary forms of these methods and the challenges they face, it is instructive to summarize key milestones in their development. Here we follow basic algorithms without mentioning additional enhancements, such as block strategies, algorithms developed for parallel processing or for use within domain factorization or a multilevel framework. We also omit the pivoting techniques based on computing some auxiliary criteria for the factorization (like minimum discard strategy), mentioning them only when needed for modifying dropping 1 strategies. We concentrate on incomplete factorizations of symmetric positive-definite systems. Since some of the related ideas have been developed in the more general framework of incomplete LU factorizations for preconditioning non symmetric systems, we will refer to them as appropriate. 2.1 Early days motivated by partial differential equations The development of incomplete factorizations closely accompanies progress in the derivation of new computational schemes, especially for partial differential equations (PDEs) that still often motivate simple theoretical ideas as well as attempts to achieve fast and robust implementations, even when the incomplete factorizations are computed completely algebraically. Whilst there is a general acceptance that the solution of some discretized PDEs requires the use of physics-based or functional space-based preconditioners, the need for reliable incomplete factorizations is widespread. They can be used not only as stand-alone procedures but they are also useful in solving augmented systems, smoothing intergrid transformations and solving coarse grid systems. The earliest incomplete factorizations were described independently by several researchers. In the late 1950s, in a series of papers [20, 21] (see also [22]), Buleev proposed a novel way to solve systems of linear equations arising from two- and three-dimensional elliptic equations that had been discretized by the method of finite differences using five- and seven-point stencils, respectively. Buleev's approach was to extend the sweep method for solving the difference equations from the one-dimensional case. To do this efficiently, he modified the system matrix so that it could be expressed in the form of a scaled product of two triangular matrices. In contemporary terminology, these two matrices represent an incomplete factorization. A slightly different and more algebraic approach to deriving an incomplete factorization can be found in the work of Oliphant [89] in the early 1960s. Starting from a PDE model of steady-state diffusion, he obtained the incomplete factorization of a matrix A corresponding to the five-point stencil discretization by neglecting the fill-in during the factorization that occurs in sub-diagonal positions. An independent algebraic derivation of the methods of Buleev and Oliphant and their interpretation as instances of incomplete factorizations was given by Varga [115]. Varga also discussed