Scholars' Mine
Masters Theses Student Theses and Dissertations
1964
The effect of matrix condition in the solution of a system of linear algebraic equations.
Herbert R. Alcorn
Follow this and additional works at: https://scholarsmine.mst.edu/masters_theses
Part of the Applied Mathematics Commons Department:
Recommended Citation Alcorn, Herbert R., "The effect of matrix condition in the solution of a system of linear algebraic equations." (1964). Masters Theses. 5642. https://scholarsmine.mst.edu/masters_theses/5642
This thesis is brought to you by Scholars' Mine, a service of the Missouri S&T Library and Learning Resources. This work is protected by U. S. Copyright Law. Unauthorized use including reproduction for redistribution requires the permission of the copyright holder. For more information, please contact [email protected]. THE EFFECT OF MATRIX CONDITION IN THE SOLUTION OF A SYSTEM OF LINEAR ALGEBRAIC EQUATIONS
BY
HERBERT RICHARD ALCORN
A
THESIS
submitted to the faculty of the
UNIVERSITY OF MISSOURI AT ROLLA in partial fulfillment of the requirements for the
Degree of
MASTER OF SCIENCE, APPLIED MATHEMATICS
Rolla, Missouri
1964
Approved by
(advisor) 11
ABSTRACT
The solution of a system of linear non-homogeneous equations may contain errors which originate from many sources. A system of linear equations in which small changes in the coefficients cause large changes in the solution is unstable and the coefficient matrix is ill- conditioned .
The purpose of this study is to define several measures of matrix condition and to test them by correlation with a measure of the actual errors introduced into a system of equations.
The study indicates that three of the five measures of condition tested were reliable indices of the magnitude of error to expect in the solution of a system of linear equations. iii
ACKNOWLEDGMENT
The author wishes to express his sincere appreciation to Professor Ralph E. Lee, Director of the Computer Science
Center for his help in the selection of this subject and for guidance and supervision during the investigation. IV
TABLE OF CONTENTS
Page
ABSTRACT ...... ii
ACKNOWLEDGMENT ...... iii
I. INTRODUCTION ...... 1
II. REVIEW OF LITERATURE ...... 3
III. D I S C U S S I O N ...... 25
IV. CONCLUSIONS ...... 32
A P P E N D I X ...... 35
BIBLIOGRAPHY ...... 37
V I T A ...... 39 1
I. INTRODUCTION
Systems of linear non-homogeneous equations arise from many sources; physical problems, numerical solution of ordinary and partial differential equations, curve fitting, data reduction, solution of the eigenvalue problem, and many others.
There are two categories of numerical solutions for systems of linear algebraic equations: exact and iterative methods. The exact method is one which will complete the solution in a known, finite number of basic arithmetic operations. An iterative solution is a means of determining an approximate solution to the system. Many of the conditions which affect the solution of the system of equations by the exact method also affect the solution by an iterative technique; however, only the exact method will be used or considered in this investigation.
Errors in the solution of a system of linear equations may arise from several sources. The need to round off numbers during the computation and the disappearance of significant figures due to the subtraction of two nearly equal quantities both contribute to error in the solution.
Also, due to physical limitations the coefficients of the equations may only be known to some degree of acci
A system of linear equations in which small changes in the coefficient matrix cause large changes in the solution 2
is unstable and shall be defined as ill-conditioned. It is the purpose of this study to define several measures of matrix condition and to test them by correlation with the effect of actual errors introduced into a system of equations. 3
II. REVIEW OF LITERATURE
There is extensive literature pertai.ning to the subject of simultaneous linear equations and to the difficulties in solving them. Consequently this survey of the literature will be presented in three parts.
A. Sources of error in the solution
D. K. Faddeev and V. N. Faddeeva (_1)* have shown the error in an element of the inverse of a matrix to be a function of the magnitude of the elements of the inverse matrix and the errors in the original matrix. From the identity
A A _1 = I , upon taking the partial derivative with respect to the ele ment of A in the i-th row and the j-th column, it follows
that
dA + A 0 , a
from which
(2 .0 1 )
*A11 numbers (x) refer to the bibliography while the numbers (x.y) refer to equations. where L e.. ij J is a zero matrix except for the element in the i-th row and j-th column which is equal to unity. Using thi definitionj
-1 dA (2.02) da ij however,, may be expressed as the product of two vectors
where - 0 0
• 1
6 and
o
Consequently 5
and letting
then
a li
-1 a2i a . i j 1 °j2 a . m
ou . a,. au .a. • • • a, .a. li j l li j 2 li jn a„.a. i a_ . a. „ • • • a„ .a. 2i j l 2i j2 2i jn
a . a. - a .a.„ ...a . a. ni j l ni j2 ni jn and therefore
•a, . a . ki jr (2 .03 )
Equation (2.03) shows that the change in each element of the inverse of A produced by a change in an element of A is the product of this change and two elements of the inverse. Thus if the inverse contains some large elements, a small or insignificant change in an element of the matrix can result in large deviations in certain elements of the inverse.
Now taking into account all of the elements of the matrix A in which changes will affect the element in the k-th row and r-th column of the inverse;
n n da. 2 2 a (2.04) kr i=l j=l
From this relationship (2.04) it can be observed that the k, r-th element of the inverse is affected by each error in A, by the magnitude of the elements of the k-th row of A and by the magnitude of the elements of the r-th column of A \
Of course, there are cases when errors due to changes in
different elements of the matrix may combine so as to compen
sate for each other.
A system of linear equations with an unstable or ill-
conditioned coefficient matrix would be unstable, since the
solution would be greatly affected by changes in the constant vector as well as in the coefficient matrix. The extent of
this instability has been noted by Hildebrand (2_)} Faddeev and Faddeeva (1_), and others. Let
Ax = b (2.05) be a system of linear equations * then
x = A ^b (2.06) and as before, Using equations (2.01), (2.02), and (2.06)
dx - A -1 A _1b da. . d a . . ij
-1 = - A e . . x . iJJ
-1 = - A L e iiJL . e-, ij . J x
“ h a2i 0 0 0
a . ni
- — and a-, .x. a li ii j dx .x. a2i x . = - 2i J J Saij a . a .x. ni m J — — _ _ from which
dx. a, . x. . kl J Sau
Similarly from equation (2.06) 8
8x __ A ~1 8b 8bi 8b.
0 M J °21 i = * a . ni 0 from which it follows that
axk a. ki (2.08)
From equations (2.07) and (2.08) it can be seen that if the inverse has large elements, then a small change in either the coefficients or in the constants can cause significant errors in the result.
From equations (2.07) and (2.08) an expression can be obtained which takes into account all of the sources of error in x;
n n n 2 2 a. . x . da. . + 2 ct. . db . ) ki J iJ ki l i=l 1 = 1 i=l and by rearranging, n n dx, = 2 a. . db. - 2 a, . x . da. . k i=i ^ ki 1 j=i ki j x j
n n = 2 a. . db . - 2 x .da . . i=i kl 1 j u j 9
Hildebrand (2) writes this as
n 5xk = ^ “ki^i (2.09) where
ri. = 6b. - (xn5a.n + x^8a.rt + •• + x 8 a . ) . (2 .1 0 ) 'i l v 1 ll 2 i2 n m ’
Equation (2.09) may be written as
5x = A Hi , where h = ^i or
A6x = h (2 .11) which may be solved simultaneously with equation (2.05) by augmenting matrix x with matrix 5x and augmenting matrix b with matrix h, or shown in partitioned form:
x . ;5x. b . .* h. 1. 1 l. l
In practice, it is usually known only that the errors
6a^ and 8b^ do not exceed some known magnitude, e; thus
- e < 5a.. < € ij - and
- e < 5b. < e . (2 .12) — i —
From equation (2.10) it is certain that
ri. i < E r 1 — (2 .1 3 ) 10
where
E = (1 + | x-^ | + | | + • • • I | )e ; (2.14) and from equation (2.09)* it follows that
n < 2 (2.15) i=l
Thus the error in x, is related to the sum of the absolute k values of the k-th row of the inverse of the coefficient matrix and the quantity E (2.14).
Scarborough (2) illustrates this error analysis by considering the system of equations
Ax = b where
1.22 -1.32 3.96 A = 2.12 -3- 52 1.62 4.23 -1.21 1.09 and 2.12 b -1.26 3-22 in which all elements have been rounded to the number of digits given; hence e = .005- The solution of the system is 0.94385 x 1.22724 0.65365 11
with
-0.04631 -0.08274 0.29123 -1 A 0.11209 -0.38058 0.15841
0.30416 -0.10137 - 0.03692
Using equation (2.14)
3 E = (1 + 2 |x. | )e = 0.0191237 i=l 1 and from equation (2.15)
3 6xn | = 2 |a,.|E = 0.0080 1 i=l 11
| = 2 | a | E = 0.0125 d i=l
3 | 5xQ | = 2 |a3 . |E = 0.0085 . J i=l 1
Thus it is evident that the solution of the true system of equations is such that
0. 936 1 Xj < 0.952 1.215 < x0 < 1.240 C- 3 0.645 1 X3 < 0.662 which could be written as
x-^ = 0.94
x2 = 1.23 X, = O .65 with the last digit in doubt by 1 unit in each case. 12
B. Some examples of ill-conditioning
There are numerous examples of ill-conditioned systems of linear equations in the literature, the following were selected to show the effects of this instability.
Turing (4_) develops an ill-conditioned system of equations from a well conditioned one in the following manner, by considering the system of equations
1.4x + 0.9y = 2.7 -0.8x + 1.7y = -1.2 (2.16) and forming from them another set by adding one-hundredth of the first to the second, to give a new equation to replace the first
-.786x + 1.709y = -I-173 -.800x + 1.700y = -1.200 . (2 .17 )
The second set is fully equivalent to the first (2.16)* but a numerical solution of the second set involving round-off errors is quite certain to yield a less accurate solution.
The solution to either set of equations is
x = 1.82903
y = 0.15^84 .
Now, modify each set of equations slightly by adding 0.001 to the coefficient of y in the second equation, from equations (2.16) 13
1.400x + 0.900y = 2.700
-0.800x + 1 .70ly = 1.200 , and from (2 .1 7 )
-0.786x + 1.709y = -1.173 -0 .800x + 1.701y = -1.200 .
The solutions to these two sets of equations are respectively:
x = 1.82908
y = 0.154X7 and
x = 1.8^779
y = 0.15887 .
The first digit to differ from the solution to the original
set of equations is underlined; it is clear that the second
set was more sensitive to a small change in the coefficient matrix than was the first. The set of equations (2.17) was described as ill-conditioned, or at any rate, ill- conditioned with respect to the first system (2.16).
Bodewig , Todd (6_), and Faddeev and Faddeeva (1_) all borrow T. S. Wilson’s well known example in integer numbers:
Ax = b where f- 5 7 6 5" 7 10 8 7 (2 -1 8 ) 6 8 10 9 5 7 9 10 14
and
23 32 33 37_ whose solution is obviously V 1 x = 1
1 and whose determinant is det(A) = 1. The ill-condition in this system of equations is apparent from the inverse
68 -41 -17 10 -41 10 -6 A ' 1 = 25 (2.19) -17 10 5 -3
. 10 -6 -3 2_
Now, perturb this system by adding to the first element of the first row of (2. 18) an amount then it can be observed how the determinant of the matrix A is affected. Let
"5 + € 7 6 5“ 7 10 8 7
A(e) = 3 6 8 10 9 5 7 9 10
then
det A(e) = 1 + 68 e _ 15
from which it follows that for
€ = ' 1 5 =-0.015 the matrix A(e) will be singular. Unless the elements of A,
(2.18), are known within 0.02, then for practical purposes the matrix must be considered singular.
C. Some measures of matrix condition
A condition number of a matrix is a measure of the stability or condition of that matrix. A large value for a condition number usually indicates an ill-conditioned matrix.
Several formulas for calculating a measure of condition of a non-singular matrix have been proposed in the literature.
F. R. Moulton (J_) discussed the solution of a system of linear algebraic equations possessing a small determinant in
1913- He illustrated in his paper a system whose solution changed appreciably with small changes in the coefficient matrix; however, he did not describe this system as ill-conditioned, nor did he propose any measure of ill- conditioning.
John von Neumann and H. H. Goldstine (8) suggest that a possible measure of matrix condition is
where 7\(A) and p(A) are the maximum and minimum eigenvalues respectively of A. John Todd (iS), (£), (10)* (_11), and (12) 16
formalized this suggested measure of condition and published a series of papers describing various properties of P(A). He also applied P(A) to a nonsymmetric matrix B by letting
A = BTB .
Todd (6_) uses (2.18) to show the effectiveness of P(A); one would expect this condition number to be large in view of the preceeding analysis, and in the case of (2.18)
P(A) ^ 3000 .
A. M. Turing (4) has proposed two additional measures of condition:
1) N-number = i N(A)N(A ^) (2.21) where ' n n il/2 N (A) = Z 2 a; (2.22) Li=l j=l and
2) M-number = n M(A)M(A ^) (2.23) where
M(A) = t>i |a..| . (2.24)
Turing adds that there is substantial agreement between these two measures, although the M-number tends to yield a larger result especially with diagonal matrices, or matrices with diagonal dominance. 17
E. Bodewig (^) defines the condition number
n
J i a i det (A)
This should be an effective measure of matrix condition pro viding there is diagonal dominance. This condition number is simpler to calculate than some of the other measures in that neither the eigenvalues nor the inverse of the matrix are required.
Andrew D. Booth (1^) agrees with von Neumann and
Goldstine that
T i*j(A ) P(A) m -n i v a ) is an effective measure of the condition of the matrix A.
Unfortunately, the calculation of the eigenvalues is usually a task of at least the complexity of the solution of the system of equations themselves, so this measure is not too practical. He adds, that if each equation is normalized by dividing that equation by
" n p il/2 2 af. , i = 1, 2, •••, n, (2.26) Lj=l then 18
n n 2 1/2 7r Z a 1 i=l .L Jbl (2.27) det(A ) det(A) v n 7 where A is the normalized matrix A, should be an effective n measure of the matrix’s condition. As an example of the practical value of his measure, Booth uses the much quoted
(2.18) set of equations * in which no ill-conditioning is evident from observation of the determinant; as in this case
det(A) = 1 .
However,, upon calculation of (2.27) we have
------^ 50,000 det(A ) x n 7 which clearly indicates the degree of ill-condition.
It has been noticed that by writing (2.27) as
n 1/2 n 2 7T Z av . J .J=1 (2.28) det(A ) det(A) v n 7 that this measure of condition is Hadamard’s inequality
n n 0 1/2 det(A) < 7T 2 af *-i=l i=l ^ divided by the actual value of the determinant. 19
In a discussion on the effect of noise on the solution of large linear systems of equations, C. Lanczos (14) defines
max A i (ATA) i min (2.29) A i (ATA) i to be a "critical ratio", and he states that any linear 2|. system whose critical ratio surpasses 10 can hardly be considered adequate for full determination of the unknowns of the problem. This condition number (2.29) has the advantage that the matrix A A is symmetric positive definite, therefore all of the eigenvalues are real and positive. It is also well known, (ljj), that if A is a symmetric matrix p and has an eigenvalue A, then A must be an eigenvalue of T A A.
J. H. Wilkinson (_16), {]J_) uses the matrix norm to develop some condition numbers. A short digression will be made at this point to define the vector and matrix norms and to state some of their properties.
A norm is an overall assessment of the magnitude of a vector or a matrix and possesses some useful properties. 20
A. The vector norm.
The norm of a vector x will be denoted by ||x || and will satisfy all three of the following conditions:
||x|| > 0 unless x = 0 (2 .30 ) |]kx|| = | k| ||x || k is a complex scalar (2 .3 1 )
lx+yII i IMI + llyl! (2 .32 )
From the second two of these conditions, it can be shown that llx -y|| ± I llx ll - llyl! I • (2.33)
The three vector norms in common use are defined by:
*llp + lx2 ip + ••• + lxn |p )1,/p (p= l»2 (2.31*)
n llx 11! = 2 x ll (2.35) i=l
" n 1/2 pd. llxll2 = S x . (2.36) -i=l
||x|| = m aX X. I • 11 00 1 1 1 (2.37)
B. The matrix norm
The norm of a square matrix A will be denoted by ||A|| and will satisfy O A unless A = 0 (2.38)
||kA|| = |k| ||A11 k is a complex scalar (2.39)
l|A+B || < IIA || + ||B|| (2.40) 21
and
l|AB |! 1 ||A || ||B I! . (2.41)
There are several matrix norms in popular use, corresponding to the three vector norms,, there are:
l|A||x = mfx s |a I (2.42) 1 J i=l 1J
n max 2 i (2.43) j=i and
max \ ( A TA) (2.44) i
The last of these, ||A||^ is known as the spectral norm, and it can be shown that if A is the unit matrix, then ||A|| = 1.
There is one additional important norm, the Euclidean or
Shur norm which is consistent with ||x|L and is defined as
' n n 2 2 (2.45) Li=l j=l
Consider the sensitivity of the solution of the set of equations
Ax = b (2.46) to variations in b. If
A(x+h) = b+k 22
then
Ah = k
and
h = A " 1k which after taking the norms yields
I N = l|A"1k|| < [|A-1 1| ||k|| (2.47)
and
which is consistent with (2 .08 ).
Now consider the relative change ||h||/||x||, from (2.46)
IN I = l|Ax|| < ||A || ||x|| (2.48)
l|x|| > ||b || llAlf1 (2.49) using (2.47)
Ih l < Mllli M 1 x[ " INI'1 l|b || and finallyy
llh 1 ||x 1\ l IIAII ||A-11| ]||- , (2.50)
in which, |[A|| ||A ^|| is the decisive quantity and may be regarded as a condition number. Using (2.44), the third matrix norm (in the notation of Faddeev and Faddeeva (!_)); 23
H-number = ||A|| ||A ^||
max (a t a ) i 1/2 _min (2.51) (ATA ) j i \
Using the Euclidean or Shur norm,
IMIe I|a _ 1 ||e = N(A)N(A_1) and from (2.21)
N-number = ^ ||A||e ||A 1 ||e . (2.52)
Richard S. Varga (18) defines the formula
A = ||S|| ||S_1 || (2.53) in which S is defined by
A = S _1JS , (2.54) where J is the Jordan normal form (or Jordan canonical form) of A. The major drawback to the use of this condition number is the extreme difficulty of calculating the simi larity transformation matrix S.
There exists considerable additional literature on the subject of condition numbers and their application, several other articles on this subject are mentioned below.
The investigation of various measures of condition was the subject of a paper by J. D. Copeland (19). 24
The establishment of a confidence region for the solution of a system of linear equtions in which the error could be considered multinormally distributed was the subject of a paper by G. E. P. Box and J. S. Hunter (20).
J. D. Calton (21) investigated various measures of condition for small matrices and proposed a measure which he thoroughly studied and reported in his paper.
There exists some interesting and informative inter relationships and inequalities between several of the condition numbers mentioned herein:
1/2 1. H-number p (a t a )
2. If A is symmetric,
P(A) = H-number o 3. N-number <_ M-number <_ (n ) N-number
4. N-number <_ H-number < (n) N-number
5* P(A) H-number
6. If A is orthogonal, then
N-number = M-number = P(A) = 1 . 25
III. DISCUSSION
A computer program was written to determine the correlation of the errors in the solution of a system of linear algebraic equations with the condition of the coefficient matrix. The program generated numerous test matrices and for each of them several measures of condition were calculated. The matrix was then randomly perturbed and the resulting system of equations was solved; a measure of error in the solution was calculated; and finally,, the correlations between the experimental measure of error and the condition numbers were computed. A detailed description of the program which was written for the IBM 1620 Model II digital computer using the Fortran II language follows, the system of n linear algebraic equations being considered is
Ax = b .
The elements of the coefficient matrix A and the constant vector b were generated by using uniformly distributed psuedo random numbers in the range
0 — aij — 1
0 < b^ < 1 for i,j = l,2,***,n .
Thus the matrices A and b are arbitrary and there is the possibility that no unique solution would exist. However, the program was written so as to reject a singular A matrix. 26
The following measures of matrix condition were included in the experimentation.
N-f-number = i N(A)N(A”^) = C-^ (3.01)
M-number = n M(A)M(A~^) ( 3 - 02)
n IT a. . i=l 11 H = = C. (3.03) det (A)
n r p il/2 T 2 a. . i=lL = C (3-04) 4 det (A ) det (A) v norm'
mf (ATA) ">l/2 H-number = (3-05) - m m '5 * i L X. (A A)
Three of the condition number formulas mentioned in the previous chapter were not included in the program. Equation
(2.29), Lanzcos "critical ratio" was not used because of its
similarity to the H-number (3-05)• Since the program is not restricted to symmetric coefficient matrices, P(A), equation
(2.20) was not used due to the difficulty in obtaining the
eigenvalues of non-symmetric matrices; and as noted previously
in the case where the coefficient matrix A is symmetric
P(A) = H-number .
The condition number formula (2.53) mentioned by Varga was not considered due to the difficulty in computing the
transformation matrices required. 27
Each element of the matrix A -was perturbed by adding to it some quantity 5a.. to form
+ ( 3 -06)
The values of 5a.. -were randomly selected from a uniform distribution in the interval
- € < 5 3 ^ < e , where the value of e was specified as data input to the program. In the same manner, each element of the dependent variable vector b -was perturbed by adding to it some random quantity 5b^, selected in the same interval as 5a_^j, to form
r i (3-07) bp = |_bi + 5bi
The perturbed system of linear equations
A b (3-08) P P was solved in order to obtain the error in each element of
the solution
5xi = xpi - xi, for i = 1,2,. (3*09)
The maximum of these errors was then used to compute an
experimental measure of condition for the coefficient matrix
max I 6x. (3-10) Ax = n n ^2 S Z I 6a.. n . i . -i i=l j=l J
which is a measure of the relative change in the solution. 28
Numerous subroutines written by the author and contained in the Computer Science Center subroutine library were utilized by the program.
RAND
This subroutine computes one of a set of uniformly distributed psuedo random numbers between zero and one.
ZERO
Each element of a matrix or a vector may be set to equal zero by the use of this subroutine.
EIGVAL
This subroutine determines the largest eigenvalue and the corresponding eigenvector of the given symmetric matrix by an iterative technique using the well known dominant eigenvalue method (I5). Evaluation of the H-number (3 .O5) required both the T largest and the smallest eigenvalues of A A, the first of these was obtained by a direct application of the subroutine to the matrix A A. Since the inverse of the matrix A was available,, having been required for other purposes, the computation of the smallest eigenvalue of A A was performed using the results which follow.
It is well known that the reciprocal of the maximum eigen value of the inverse of the matrix is the minimum eigenvalue 29
of the matrix itself. Instead of inverting the matrix T A A, the following is done: let
T B = A lA (3-11) then
-1 B (a ’a )-1
= A - h A 1 )-1 = A ' V V (3 .12)
Using equation (3*12) and the basic equation
Bx = xx (3-13) it follows that
A 1 (A V x = kx (3-14) where
k = I . (3.15)
Thus the application of the subroutine EIGVAL to the matrix -1 - I T A (A ) would yield the reciprocal of the smallest eigen- T value of A A.
INVRT
This procedure uses the method of Gauss-Jordan elimina tion, pivoting on the maximum element in the matrix,, to compute the inverse and the determinant of the given matrix.
MTXMPY
The multiplication of two matrices,, a matrix and a vector, or two vectors is performed with this subroutine. 30
AMAX
This subprogram finds the maximum value of a given set of numbers,, either the element of greatest algebraic value or the element of maximum absolute value may be found.
GAUJOR
This subroutine solves a given set of n linear algebraic equations by the method of Gauss-Jordan reduction, pivoting on the element of maximum magnitude by columns is employed to help retain accuracy in near singular systems. This routine does not solve a set of equations whose coefficient matrix is singular.
Experimentation using the previously described program was performed in the following manner. The input variables to the program consisted of n, the size of the system of equations,* e, the bound upon the random perturbation of the system; and m, the number of times this particular system is to be perturbed and solved. For each perturbation and solution, the quantity Ax is calculated, these are averaged at the conclusion of the m trials, and define the quantity
i m A = - 2 Ax, . (3-16) m k=l K
The size of the system, n, was allowed to vary from 2 -6 to 35; for each size three values of e were used, 10 ,
- 4 - 2 10 , and 10 ; for each value of e five trials were conducted. The output of the program for each separate value of e was: 31
1. n, the size of the system
2. e, the bound on the perturbation
3. A, the experimental measure of error
4. V i = 1,2,••*,5 the five condition numbers
After extensive experimentation with the program, coefficients indicating the degree of correlation between the experimental measure of error and the five condition numbers were calculated as follows. Let r^ be the simple product-moment correlation coefficient (22) between the experimental measure of error and the condition number Ch, then
2A.th C it . „ --- m 2a +_ t 2C. it. r . r (3-17) ,/i - k ^2 scit - Ac = a C± + b (3.18) where the coefficients a and b are to be determined by the method of least squares. The range of values for r^ is -1 to +1; r. = 0 indicates a total lack of correlation between the variables, r^ = +1 or -1 indicates perfect positive or negative linear correlation. 32 IV. CONCLUSIONS From the results of the experimentation and the corre lation of these results, several conclusions were drawn. The most important is that all of the condition formulas tested, with the exception of two, seem to be reliable indicators of the magnitude of error to expect in the solution of a system of linear non-homogenous equations. Three of the formulas investigated yield condition numbers which appear to be well correlated* with the experimental measure of condition (3 .16 ), they are: N-number = — N(A)N(A = C-^ M-number = n M(A)M(A = C0 ■mfx X (a t a )'1/2 H-number = = C, • m m T a i x. (A A) Of these three formulas, and C^ are slightly better corre lated with A (3*16) than is C^, also the correlation coeffi cients obtained for C-^ and C^ with A are nearly equal in all cases. This is a desirable conclusion in that a value for Cf or C^ is considerably easier to compute than is a value for Cc . *The appendix contains tabulations of the correlation coefficients obtained. 33 The two remaining condition formulas studied n 7T a. . i=l 11 u C det (A) 3 and 1/2 n n 2 TT 2 aT. i=lLi=l = C, d e t(An o rJ I |det(A)| seem to be poor measures of a matrix1s condition in that there is little correlation between them and the experimental measure of condition A. However,, when the size of the system of equations is held constant, or nearly so, and appear to be better indicators of condition than when the size is allowed to vary. It was noticed that as the size of the system of equations was increased, the values obtained for became smaller and those obtained for increased; this is an undesirable characteristic for a condition number to possess. The values observed for the correlation coefficients _ 6 for Cp Cgj and versus A when e = 10 were nearly identical to those observed for the same Ch when e = 10 ; _p however, when e = 10 (approximately one per cent of the value of the elements of the matrix), there was less correlation for the same CL. With additional experimentation this observation might lead to the conclusion that as the 34 error in an element the matrix approaches the magnitude of that element, the condition numbers C-^, C^, and become less indicative of the errors in the solution vector x. It was also observed that small correlation coeffi cients were obtained for some values of n, the following is a possible explanation of this phenomenon. If a set of observations of and A were to be plotted, and the matrices considered were of approximately the same condition; most of the points plotted would lie in a small cluster and could not define a straight line as well as a more widely scattered set of points. In summary, it could be stated that for small values of e, the condition number formulas C-p C^, and should provide reliable indices of the condition of the coefficient matrix and the errors in the solution of system of equations. 35 APPENDIX The following three tables contain tabulations of the correlation coefficients obtained between the condition numbers Ch., i = 1,2,•••,5 an^ the experimental measure of matrix condition A for various arrangements of the data. TABLE 1. Correlation between condition numbers and experimental measure of condition for all systems of equations tested. Size of B ound on Number of System Pertur Systems Correlation Coefficients bation Run n e m r l r2 r3 r5 10'6 2-35 10-4 981 .819 .815 .010 -.003 • 770 10-2 TABLE 2. Correlation between condition numbers and experimental measure of condition for the three perturbation bounds. n € m r i r2 r3 r4 r 5 CO 1 -1 I- O 2-35 327 .919 .914 .006 -.004 .860 2-35 10"4 327 •927 .923 .006 -.004 .867 i ro i—‘ 2-35 o 327 • 557 .555 .025 -.007 •535 36 TABLE 3- ween condition numbers and experimental mea ition for the various sizes of systems tes 10-6, 10'\ and 10'2. n r l r2 r3 r4 r5 2 . 946 .925 .964 .945 • 945 3 • 853 .869 . 568 .754 • 855 4 • 913 .960 .858 .828 .914 5 . 661 .668 .621 .710 . 666 6 • 797 . 704 -.066 .253 • 779 7 .987 •992 .611 .676 .986 8 .701 • 732 .416 .243 .699 9 .864 .864 . 142 .849 .864 10 .917 .980 • 544 • 591 .914 11 .774 • 756 .080 .788 • 773 12 .347 .372 -.013 . 150 .341 13 .828 .820 • 835 .499 .827 14 .554 .547 . 520 . 584 • 553 15 .894 .887 • 345 • 949 .893 16 • 959 .966 .964 .908 .960 17 .919 .917 .884 .917 .919 18 .506 .787 .366 .162 . 544 19 .921 • 924 .951 .950 .920 20 - .978 .912 -.320 -.303 .923 . 806 .820 .530 • 635 .804 37 BIBLIOGRAPHY 1. Faddeev, D. K. and Faddeeva, V. N., (1963) Computational Methods of Linear Algebra Translated by Robert C. Williams, W. 1L Freeman and Co., San Francisco and London, p. 119-128. 2. Hildebrand, F. B., (1956) Introduction to Numerical Analysis, McGraw-Hill, New York, p . 436-439 . 3- Scarborough, J. B., (1962) Numerical Mathematical Analysis, 5th Ed., The John Hopkins Press, Baltimore, p. 301-305. 4. Turing, A. M., (1948) Rounding-Off Errors in Matrix Processes, Quarterly Journal of Mechanics and Applied Mathematics, 1: p7 287- 3O8 . 5 . Bodewig, E., (1959) Matrix Calculus, 2nd Ed., Interscience Publishers, Inc., New York, p. 133-137* 6 . Todd, John, (1962) Survey of Numerical Analysis, McGraw Hill, New York, pi 239-243. 7 . Moulton, F. R., (1913) On the Solutions of Linear Equations Having Small Determinants, American Mathematical Monthly, 20: p. 242-249. 8 . Von Neumann, John and H. H. Goldstine, (1947) Numerical Inverting of Matrices of High Order, Bulletin of the American Mathematical Society, 53* p. 1021-1099. 9. Todd, John, (1954) The Condition of the F in ite Segments of the Hilbert Matrix, National Bureau of Standards Applied Mathematics Series, 39: p . 109-116. 10. Todd, John, (1949) The Condition of Certain Matrices, I, Quarterly Journal of Mechanics and Applied Mathematics, 2: p. 469-472“ 11. Todd, John, (1950) The Condition of a Certain Matrix, P roc. of the Cambridge P hilosophical S o cie ty , 46: __ pq7J7TT87 38 12. Todd, John* (1958) The Condition of Certain Matrices, III, Journal of Research National Bureau of Standards,, 60: p. 1-7. 13. Booths Andrew D., (1957) Numerical Methods, 2nd Ed., Academic Press Inc., New York, pi 80-85. 14. Lanczos, Cornelius, (1956) Applied Analysis, Prentice- Hall, Inc., Englewood Cliffs, N. J ., p . 167-170. 15* Hildebrand, F. B., (1952) Methods of Applied Mathematics, Prentice-Hall, Inc., Englewood Cliffs, N. J ., p7 1-95• 16. Wilkinson, J. H., (1963) Rounding Errors in Algebraic Processes, Prentice-Hall, Inc., Englewood Cliffs, N. J., p. 79-93. 17 . Wilkinson, J. H., (1961) Error Analysis of Direct Methods of Matrix Inversion, Journal of the Association of Computing Machinery, 8: p . 281-330. 18. Varga, Richard S., (1962) Matrix Iterative Analysis, Prentice-Hall, Inc., Englewood Cliffs, N. J ., p . 66, 95* 19. Copeland, J. D., (1963) On Condition Numbers for Matrices. Thesis, University of Texas, 5O P • 20. Box, G. E. P. and J. S. Hunter, (1954) A Confidence Region for the Solution of a Set of Simultaneous Equations with an Application to Experimental Design, Biometrika, 41: p. 190-199. 21. Calton, T. D., (1963) Investigation of Measures of I11-Conditioning. Thesis, Missouri School of Mines and Metallurgy, 40 p . 22. Ralston, Anthony and Herbert S. Wilf, (I960) Mathematical Methods for Digital Computers, John Wiley and Sons, New York, pT 213-220. 39 VITA The author was born October 2 4 , 1933, at St. Louis, Missouri. His primary education was received there, he attended high school in Kirkwood, Missouri, graduating in June 1953. He received a Bachelor of Science degree in Mechanical Engineering from the University of Missouri School of Mines and Metallurgy at Rolla, Missouri, in June 1962. In September 1962, work was started toward the Master of Science degree in Applied Mathematics. Since July 1962, the author has been employed as a Computer Analyst and Instructor in Computer Science by the University of Missouri School of Mines and Metallurgy Computer Science Center.