Scholars' Mine Masters Theses Student Theses and Dissertations 1964 The effect of matrix condition in the solution of a system of linear algebraic equations. Herbert R. Alcorn Follow this and additional works at: https://scholarsmine.mst.edu/masters_theses Part of the Applied Mathematics Commons Department: Recommended Citation Alcorn, Herbert R., "The effect of matrix condition in the solution of a system of linear algebraic equations." (1964). Masters Theses. 5642. https://scholarsmine.mst.edu/masters_theses/5642 This thesis is brought to you by Scholars' Mine, a service of the Missouri S&T Library and Learning Resources. This work is protected by U. S. Copyright Law. Unauthorized use including reproduction for redistribution requires the permission of the copyright holder. For more information, please contact [email protected]. THE EFFECT OF MATRIX CONDITION IN THE SOLUTION OF A SYSTEM OF LINEAR ALGEBRAIC EQUATIONS BY HERBERT RICHARD ALCORN A THESIS submitted to the faculty of the UNIVERSITY OF MISSOURI AT ROLLA in partial fulfillment of the requirements for the Degree of MASTER OF SCIENCE, APPLIED MATHEMATICS Rolla, Missouri 1964 Approved by (advisor) 11 ABSTRACT The solution of a system of linear non-homogeneous equations may contain errors which originate from many sources. A system of linear equations in which small changes in the coefficients cause large changes in the solution is unstable and the coefficient matrix is ill- conditioned . The purpose of this study is to define several measures of matrix condition and to test them by correlation with a measure of the actual errors introduced into a system of equations. The study indicates that three of the five measures of condition tested were reliable indices of the magnitude of error to expect in the solution of a system of linear equations. iii ACKNOWLEDGMENT The author wishes to express his sincere appreciation to Professor Ralph E. Lee, Director of the Computer Science Center for his help in the selection of this subject and for guidance and supervision during the investigation. IV TABLE OF CONTENTS Page ABSTRACT .............................................. ii ACKNOWLEDGMENT ........................................ iii I. INTRODUCTION ................................... 1 II. REVIEW OF LITERATURE .......................... 3 III. D I S C U S S I O N ..................................... 25 IV. CONCLUSIONS ................................... 32 A P P E N D I X ............................................... 35 BIBLIOGRAPHY .......................................... 37 V I T A ................................................... 39 1 I. INTRODUCTION Systems of linear non-homogeneous equations arise from many sources; physical problems, numerical solution of ordinary and partial differential equations, curve fitting, data reduction, solution of the eigenvalue problem, and many others. There are two categories of numerical solutions for systems of linear algebraic equations: exact and iterative methods. The exact method is one which will complete the solution in a known, finite number of basic arithmetic operations. An iterative solution is a means of determining an approximate solution to the system. Many of the conditions which affect the solution of the system of equations by the exact method also affect the solution by an iterative technique; however, only the exact method will be used or considered in this investigation. Errors in the solution of a system of linear equations may arise from several sources. The need to round off numbers during the computation and the disappearance of significant figures due to the subtraction of two nearly equal quantities both contribute to error in the solution. Also, due to physical limitations the coefficients of the equations may only be known to some degree of acci A system of linear equations in which small changes in the coefficient matrix cause large changes in the solution 2 is unstable and shall be defined as ill-conditioned. It is the purpose of this study to define several measures of matrix condition and to test them by correlation with the effect of actual errors introduced into a system of equations. 3 II. REVIEW OF LITERATURE There is extensive literature pertai.ning to the subject of simultaneous linear equations and to the difficulties in solving them. Consequently this survey of the literature will be presented in three parts. A. Sources of error in the solution D. K. Faddeev and V. N. Faddeeva (_1)* have shown the error in an element of the inverse of a matrix to be a function of the magnitude of the elements of the inverse matrix and the errors in the original matrix. From the identity A A _1 = I , upon taking the partial derivative with respect to the ele­ ment of A in the i-th row and the j-th column, it follows that dA + A 0 , a from which (2 .0 1 ) *A11 numbers (x) refer to the bibliography while the numbers (x.y) refer to equations. where L e..ij J is a zero matrix except for the element in the i-th row and j-th column which is equal to unity. Using thi definitionj -1 dA (2.02) da ij however,, may be expressed as the product of two vectors where - 0 0 • 1 6 and o Consequently 5 and letting then a li -1 a2i a . i j 1 °j2 a . m ou . a,. au .a. • • • a, .a. li j l li j 2 li jn a„.a. i a_ . a. „ • • • a„ .a. 2i j l 2i j2 2i jn a . a. - a .a.„ ...a . a. ni j l ni j2 ni jn and therefore •a, . a . ki jr (2 .03 ) Equation (2.03) shows that the change in each element of the inverse of A produced by a change in an element of A is the product of this change and two elements of the inverse. Thus if the inverse contains some large elements, a small or insignificant change in an element of the matrix can result in large deviations in certain elements of the inverse. Now taking into account all of the elements of the matrix A in which changes will affect the element in the k-th row and r-th column of the inverse; n n da. 2 2 a (2.04) kr i=l j=l From this relationship (2.04) it can be observed that the k, r-th element of the inverse is affected by each error in A, by the magnitude of the elements of the k-th row of A and by the magnitude of the elements of the r-th column of A \ Of course, there are cases when errors due to changes in different elements of the matrix may combine so as to compen­ sate for each other. A system of linear equations with an unstable or ill- conditioned coefficient matrix would be unstable, since the solution would be greatly affected by changes in the constant vector as well as in the coefficient matrix. The extent of this instability has been noted by Hildebrand (2_)} Faddeev and Faddeeva (1_), and others. Let Ax = b (2.05) be a system of linear equations * then x = A ^b (2.06) and as before, Using equations (2.01), (2.02), and (2.06) dx - A -1 A _1b da. d a . ij -1 = - A e . x . iJJ -1 = - A L eiiJL . e-,ij . J x “ h a2i 0 0 0 a . ni - — and a-, .x. a li ii j dx .x. a2i x . = - 2i J J Saij a . a .x. ni m J — — _ _ from which dx. a, . x. kl J Sau Similarly from equation (2.06) 8 8x __ A ~1 8b 8bi 8b. 0 M J °21 i = * a . ni 0 from which it follows that axk a. ki (2.08) From equations (2.07) and (2.08) it can be seen that if the inverse has large elements, then a small change in either the coefficients or in the constants can cause significant errors in the result. From equations (2.07) and (2.08) an expression can be obtained which takes into account all of the sources of error in x; n n n 2 2 a. x . da. + 2 ct. db . ) ki J iJ ki l i=l 1 = 1 i=l and by rearranging, n n dx, = 2 a. db. - 2 a, . x . da. k i=i ^ ki 1 j=i ki j x j n n = 2 a. db . - 2 x .da . i=i kl 1 j u j 9 Hildebrand (2) writes this as n 5xk = ^ “ki^i (2.09) where ri. = 6b. - (xn5a.n + x^8a.rt + •• + x 8 a . ) . (2 .1 0 ) 'i l v 1 ll 2 i2 n m ’ Equation (2.09) may be written as 5x = A Hi , where h = ^i or A6x = h (2 .11) which may be solved simultaneously with equation (2.05) by augmenting matrix x with matrix 5x and augmenting matrix b with matrix h, or shown in partitioned form: x . ;5x. b . .* h. 1. 1 l. l In practice, it is usually known only that the errors 6a^ and 8b^ do not exceed some known magnitude, e; thus - e < 5a.. < € ij - and - e < 5b. < e . (2 .12) — i — From equation (2.10) it is certain that ri. i < E r 1 — (2 .1 3 ) 10 where E = (1 + | x-^ | + | | + • • • I | )e ; (2.14) and from equation (2.09)* it follows that n < 2 (2.15) i=l Thus the error in x, is related to the sum of the absolute k values of the k-th row of the inverse of the coefficient matrix and the quantity E (2.14). Scarborough (2) illustrates this error analysis by considering the system of equations Ax = b where 1.22 -1.32 3.96 A = 2.12 -3- 52 1.62 4.23 -1.21 1.09 and 2.12 b -1.26 3-22 in which all elements have been rounded to the number of digits given; hence e = .005- The solution of the system is 0.94385 x 1.22724 0.65365 11 with -0.04631 -0.08274 0.29123 -1 A 0.11209 -0.38058 0.15841 0.30416 -0.10137 - 0.03692 Using equation (2.14) 3 E = (1 + 2 |x.