68 Journal of Advances in Applied & Computational Mathematics, 2016, 3, 68-73 Forward Stability of Iterative Refinement with a Relaxation for Linear Systems * Alicja Smoktunowicz , Jakub Kierzkowski and Iwona Wróbel Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa 75, 00-662 Warsaw, Poland Abstract: Stability analysis of Wilkinson’s iterative refinement method IR( !) with a relaxation parameter ! for solving linear systems is given. It extends existing results for ! =1 , i.e., for Wilkinson’s iterative refinement method. We assume that all computations are performed in fixed (working) precision arithmetic. Numerical tests were done in MATLAB to illustrate our theoretical results. A particular emphasis is given on convergence of iterative refinement method with a relaxation. A preliminary error analysis of the Algorithm IR( !) was given in [11]. Our opinion is opposite to that given in [11], since our experiments show that the choice ! =1 is the best choice from the point of numerical stability. Keywords: Iterative refinement, numerical stability, condition number. 1. INTRODUCTION numerical integration of a dynamic system with step size h . A preliminary error analysis of the Algorithm We consider the system Ax = b , where A ! Rn"n is IR( !) was given in [11] for 0 < ! < 1 , assuming that nonsingular and b ! Rn . Iterative refinement the extended precision is used for computing the techniques for linear systems of equations are very residual vectors rk . Wu and Wang considered only useful in practice and the literature on this subject is Gaussian elimination as a solver S . very rich, see [1], [4]– [11]. The purpose of this paper is to analyze the The idea of relaxing the iterative refinement step is convergence of this method for 0 < ! < 2 and to show the following. We require a basic linear equation solver with examples that the choice ! = 1 is the best choice S for Ax = b which uses a factorization of A into simple from the point of numerical stability. factors (e.g., triangular, block-triangular etc.). Such factorization is used again in the next steps of iterative Notice that for arbitrary ! > 0 , the IR(! ) method is refinement. Wilkinson’s iterative refinement method a stationary method (in the theory) and we have with a relaxation IR(! ) consists of three steps. !1 * pk = A rk = x ! xk , so x ! x* = (1!")(x ! x* ), k = 0,1,…, where x* is the Algorithm IR(! ) k+1 k exact solution to Ax = b . We see that the sequence Given ! > 0 . Let x be computed by the solver S . 0 {xk } is convergent for arbitrary initial x0 if and only if 0 < ! < 2 . For ! = 1 (Wilkinson’s iterative refinement) For k = 0,1,2, , the k th iteration consists of the * … x will be the exact solution x . It is interesting to three steps: 1 check the influence on the relaxation parameter ! on numerical properties of the algorithm IR(!) , assuming 1. Compute rk = b ! Axk . that all computations are performed only in the working (fixed) precision. 2. Solve Apk = rk for pk by the basic solution solver S . Throughout the paper we use only the 2-norm and assume that all computations are performed in the 3. Add the correction, xk+1 = xk +! pk . working (fixed) precision. We use a floating point arithmetic which satisfies the IEEE floating point Clearly, ! =1 corresponds to Wilkinson’s iterative standard. For two floating point numbers a and b we refinement method [10]. Wu and Wang [11] proposed have h this method for ! = , where h > 0 (i.e., for f !(a!b) = (a!b)(1+ "), | " |# $ h +1 M 0 < ! < 1 ). They developed the method as the for results in the normalized range, where ! denotes any of the elementary scalar operations +,!,*, / and *Address correspondence to this author at the Faculty of Mathematics and Information Science, Warsaw University of Technology, Koszykowa 75, 00-662 !M is machine precision. Warsaw, Poland; Tel: +48222347988; Fax: +48226257460; E-mail: [email protected] E-ISSN: 2409-5761/16 © 2016 Avanti Publishers Forward Stability of Iterative Refinement with a Relaxation Journal of Advances in Applied & Computational Mathematics, 2016, Vol. 3, No. 2 69 In this paper we present a comparison of computed vectors in floating point arithmetic. Assume Wilkinson’s iterative refinement method with a that relaxation IR( ! ) from the point of view of numerical stability. More precisely, we say that the computed x! in !M " 0.01, L(n)!M #(A) " 0.01 (5) floating point arithmetic is a forward stable solution to Ax = b if and * * |1 | q 0.6. (6) ! x" ! x ! " O(#M )$(A) ! x ! . (1) !" +" # Throughout this paper, is the matrix or vector Then for k = 0,1,… we have ! ! ! two–norm depending upon context, and * * "1 (7) !(A) = ! A ! ! A ! denotes the standard condition || x!k ! x ||" qk || x ||, qk " 0.1, number of the matrix A . where A stronger property than forward stability is q = (|1!" | +q")q + 2.31"L(n)# $(A)+1.64# , (8) backward stability. It means that the computed ~x in k+1 k M M floating point arithmetic is the exact solution of a slightly perturbed system with q0 = q . (A + !A)x! = b, " !A "" O(# ) " A " . (2) Proof. Assume that (7) holds for k . We prove that it M holds also for k +1, i.e. || x! ! x* ||" q || x* || , where k+1 k+1 Our analysis is similar in spirit to [4]-[6]. Jankowski qk+1 ! 0.1 and qk+1 satisfies (8). and Wo z! niakowski [6] prove that an arbitrary solver S which satisfies (3), supported by iterative refinement, is Under assumption (4), the computed vectors ~r normwise forward stable as long as A is not too ill- k satisfy conditioned (say, !M "(A) < 1 ), and is normwise backward stable under additional condition q!(A) < 1. r! = b ! Ax! + "r , || "r ||# $ L(n)(|| b || + || A |||| x! ||). (9) k k k k M k We extend their results for the algorithm IR(!) , see Theorems 2.1. Under assumption (3) we have The paper is organized as follows. A proof of * * "1 * p! k = pk + !pk , pk = A r!k , " !pk "# q " pk " . (10) forward stability of IR(!) is given in Section 2. In Section 3, we present some numerical experiments Standard error analysis shows that illustrate our theoretical results. (1) (2) (i) x!k+1 = (I + Dk )(x!k + (I + Dk )! p! k ), " Dk "" #M . (11) 2. FORWARD STABILITY OF IR(!) * * By inductive assertion, we have ! x"k ! x !" qk ! x ! . We require a basic linear equation solver S for Hence Ax = b such that the computed solution x! by S satisfies * * * * * ! x" !=! x + (x" ! x ) !"! x ! + ! x" ! x !" (1+ q ) ! x ! . k k k k ! x" ! x* !" q! x* !, q " 0.1. (3) Similarly, from (10) it follows that ! p" !! (1+ q) ! p* ! , thus We make a standard assumption that the matrix- k k vector multiplication is backward stable. Then the * * computed residual vector r! = f "(b ! Ax!) satisfies ! x"k !! 1.1 ! x !, ! p" k !! 1.1 ! pk ! . (12) r! = b ! Ax! + "r, " "r "# L(n)$ (|| b || + || A |||| x! ||), (4) From (9) and the inequality ! b !=! Ax* !!! A ! ! x* ! it M can be seen that where L(n) is a modestly growing function on n . r! = b ! Ax! + "r , " "r "# 2.1L(n)$ " A " " x* " . (13) k k k k M We start with the following lemma. We have Lemma 2.1 Let IR(!) for ! " (0, 2) be applied to the nonsingular linear system Ax = b using the solver p* = A!1r! = x* ! x! + " , " = A!1#r . (14) k k k k k k S satisfying (3)-(4). Let x! , r! and p! denote the k k k 70 Journal of Advances in Applied & Computational Mathematics, 2016, Vol. 3, No. 2 Smoktunowicz et al. This together with (13) implies the bounds Proof. We apply the results of Lemma 2.1. Notice that from (7)-(8) and by assumptions (5) it follows that * * * ! pk !!! x"k " x ! + ! #k !, ! #k !! 2.1L(n)$M %(A) ! x ! .(15) qk+1 ! qk 0.6 + 2.31"L(n)#M $(A)+1.64#M . * Now our task is to bound the error ! x"k+1 ! x ! . For (3) Since ! < 2 and 1 ! "(A) , we get simplicity, we define Dk such that q ! q 0.6 + (4.62L(n)+1.64)" #(A). (3) (1) (2) k+1 k M I + Dk = (I + Dk )(I + Dk ). From this it follows that (3) 2 Clearly, ! Dk !! 2"M +"M , so from (11) we get k 4.62L(n)+1.64 qk+1 ! (0.6) + #M $(A). 2 1" 0.6 x!k+1 = (x!k +! p! k )+"k , " "k "# $M " x!k " +(2$M +$M )! " p! k " . (16) From this (19) follows immediately. This together with (10) and (14) gives the identity 3. NUMERICAL EXPERIMENTS x! ! x* = (1!")(x! ! x* )+# +"($ + %p ). k+1 k k k k In this section we present numerical experiments Taking norms and using (10), we obtain that show the comparison of the IR(!) for different values of ! . All tests were performed in MATLAB * * * $16 ! x"k+1 ! x !"|1!# | ! x"k ! x ! + ! $k ! +# ! %k ! +#q ! pk ! .(17) version 8.4.0.150421 (R2014b), with !M " 2.2 #10 . * * * !1 First we estimate ! !k ! . Since ! x"k ! x !" 0.1 ! x ! , Let x = A b be the exact solution to Ax = b and let ~ * so by assumption (5) we obtain from (15) the bounds xk be the computed approximation to x by IR(!) . We produced the matrix and the vector * * * n ! n A 0.021 x , p 0.121 x . (18) * * T ! !k !" ! ! ! k !" ! ! , with . b = Ax x = [1,1,…,1] From (12) and (16) we have We report the following statistics for each iteration: * * ! !k !" 1.1#M (! x ! +(2 +#M )$ ! pk !) .