ECE 3040 Lecture 16: Systems of Linear Equations II © Prof. Mohamad Hassoun
This lecture covers the following topics:
Introduction LU factorization and the solution of linear algebraic equations Matlab’s lu function and the left-division operator “\” More on Matlab’s left-division operator Iterative methods: Jacobi and Gauss-Seidel algorithms
Introduction
In the previous lecture, Gauss elimination with pivoting was used to solve a system of linear algebraic equations of the form: 퐀퐱 = 퐝. This method becomes inefficient for solving problems where the coefficient matrix 퐀 is constant, but with different right-hand-side vectors 퐝. Recall that Gauss elimination involves a forward elimination step (which dominates the computation time) that couples the vector 퐝 and the matrix 퐀 through the augmented matrix [퐀 퐝]. This means that every time 퐝 changes we have to repeat the elimination step all over. The LU factorization-based solution method separates (decouples) the time-consuming elimination of the matrix A from the manipulations of the vector 퐝. Thus, once matrix 퐀 has been factored as the product of two square matrices, 퐀 = 퐋퐔, multiple right-hand vectors 퐝 (as they become available) can be evaluated in an efficient manner.
Note: If multiple right hand vectors 퐝1, 퐝2, …, 퐝푚 are available simultaneously, then we can still apply (efficiently) Gauss elimination to the augmented system
[퐀 퐝1 퐝2 … 퐝푚] and obtain
[퐀′ 퐝′1 퐝′2 … 퐝′푚]
The solution vectors 퐱1, 퐱2, …, 퐱푚 can then be determined, respectively, by back substitution using the systems
′ ′ ′ 퐀 퐱1 = 퐝′1, 퐀 퐱2 = 퐝′2, …, 퐀 퐱푚 = 퐝′푚.
However, the complication arises when the 퐝푖 vectors are not available simultaneously. Here, the elimination step would have to be repeated 푚 times which renders the Gauss elimination method impractical for solving large systems of equations.
LU Factorization and the Solution of Linear Algebraic Equations
The LU factorization of an 푛x푛 matrix A requires pivoting (i.e., row permutations) just as was the case for Gauss elimination. If matrix A is nonsingular (i.e., |A| ≠ 0), then it can be shown that PA = LU, where P is a permutation matrix, L is a lower triangular matrix with 1’s on the diagonal, and U is an upper triangular matrix. For simplicity, the factorization method will be described for a 3x3 matrix that requires no pivoting (i.e., P = I or A = LU), where
1 0 0 푢11 푢12 푢13 L = [푙21 1 0], U = [ 0 푢22 푢23] 푙31 푙32 1 0 0 푢33
The coefficients 푙푖푗 and 푢푖푗 can be computed from the 푎푖푗 coefficients, as follows. Set A equal to the product LU (note that, in general, matrix multiplication is not commutative LU ≠ UL):
푎11 푎12 푎13 1 0 0 푢11 푢12 푢13 [푎21 푎22 푎23] = [푙21 1 0] [ 0 푢22 푢23] 푎31 푎32 푎33 푙31 푙32 1 0 0 푢33 or,
푎11 푎12 푎13 푢11 푢12 푢13 [푎21 푎22 푎23] = [푙21푢11 푙21푢12 + 푢22 푙21푢13 + 푢23 ] 푎31 푎32 푎33 푙31푢11 푙31푢12 + 푙32푢22 푙31푢13 + 푙32푢23 + 푢33
By matching coefficients in the above matrix equation we obtain the following formulas:
푢11 = 푎11
푢12 = 푎12
푢13 = 푎13 푎21 푎21 푙21 = = 푢11 푎11 푎21푎12 푢22 = 푎22 − 푙21푢12 = 푎22 − 푎11 푎21푎13 푢23 = 푎23 − 푙21푢13 = 푎23 − 푎11 푎31 푎31 푙31 = = 푢11 푎11 푎 푎 푎 − 31 12 푎 − 푙 푢 32 푎 푎 푎 − 푎 푎 푙 = 32 31 12 = 11 = 11 32 31 12 32 푢 푎21푎12 푎 푎 − 푎 푎 22 푎22 − 11 22 21 12 푎11
푢33 = 푎33 − 푙31푢13 − 푙32푢23 푎31푎13 (푎11푎32 − 푎31푎12)(푎11푎23 − 푎21푎13) = 푎33 − − 푎11 푎11(푎11푎22 − 푎21푎12)
Regrouping the above results, gives
푢11 = 푎11
푢12 = 푎12
푢13 = 푎13 푎21푎12 푢22 = 푎22 − 푎11 푎21푎13 푢23 = 푎23 − 푎11
푎31푎13 (푎11푎32 − 푎31푎12)(푎11푎23 − 푎21푎13) 푢33 = 푎33 − − 푎11 푎11(푎11푎22 − 푎21푎12)
푎21 푙21 = 푎11 푎31 푙31 = 푎11 푎11푎32 − 푎31푎12 푙32 = 푎11푎22 − 푎21푎12 Notice that 푎11푎22 − 푎21푎12 ≠ 0 and 푎11 ≠ 0 are required for the decomposition to exist. However, if A is nonsingular and we allow row permutations, we may relax those conditions. Your turn: Derive the above formulas for a 4x4 matrix, assuming that row permutations are not necessary.
Example. Perform LU factorization for the following matrix. 1 1 1 [ 4 2 ] 1 1 1 16 4 1
푎21 1 푙21 = = = 4 푎11 1 4
푎31 16 푙31 = = = 64 푎11 1 4 푎 푎 − 푎 푎 1 − 8 푙 = 11 32 31 12 = = +28 32 1 1 푎11푎22 − 푎21푎12 − 4 2 Therefore the lower triangular matrix L is, 1 0 0 1 0 0 L = [푙21 1 0] = [ 4 1 0] 푙31 푙32 1 64 28 1 And, 1 푢 = 푎 = 11 11 4 1 푢 = 푎 = 12 12 2
푢13 = 푎13 = 1 1 (1) ( ) 푎21푎12 2 푢22 = 푎22 − = 1 − = −1 푎11 1 4
푎21푎13 (1)(1) 푢23 = 푎23 − = 1 − = −3 푎11 1 4
푎31푎13 (푎11푎32 − 푎31푎12)(푎11푎23 − 푎21푎13) 푢33 = 푎33 − − = 푎11 푎11(푎11푎22 − 푎21푎12) 1 (16)(1) (1 − 8) ( − 1) 1 − − 4 = 1 − 64 + 84 = 21 1 1 1 1 ( − ) 4 4 4 2 leading to the upper triangular matrix,
푢11 푢12 푢13 1/4 1/2 1 U = [ 0 푢22 푢23] = [ 0 −1 −3] 0 0 푢33 0 0 21 Verification:
Now, let us go back to the system Ax = d. We may rewrite it as LUx = d or L(Ux) = d. Define the intermediate vector z, as z = Ux. Then, we may write the original system as Lz = d, or explicitly as
1 0 0 푧1 푑1 [푙21 1 0] [푧2] = [푑2] 푙31 푙32 1 푧3 푑3
The above formulation can be solved by forward-substitution,
푧1 = 푑1
푧2 = 푑2 − 푙21푧1
푧3 = 푑3 − 푙31푧1 − 푙32푧2
Once z is determined, we can solve for x using back-substitution with Ux = z, or explicitly,
푢11 푢12 푢13 푥1 푧1 [ 0 푢22 푢23] [푥2] = [푧2] 0 0 푢33 푥3 푧3
The solution for x, assuming non-zero diagonal elements (푢푖푖 ≠ 0), is
푧3 푥3 = 푢33 푧2 − 푢23푥3 푥2 = 푢22 푧1 − 푢12푥2 − 푢13푥3 푥1 = 푢11 Returning to our original 3x3 A matrix and its associated LU factorization, and assuming a right-hand vector d = [2 1 8]T, we can then apply the forward- substitution step to the system
1 0 0 푧1 2 [ 4 1 0] [푧2] = [1] 64 28 1 푧3 8 to obtain the intermediate vector z,
푧1 = 푑1 = 2
푧2 = 푑2 − 푙21푧1 = 1 − 4(2) = −7
푧3 = 푑3 − 푙31푧1 − 푙32푧2 = 8 − 64(2) − 28(−7) = 76 leading to the intermediate solution, z = [2 -7 76]T. Finally, the back-substitution step is applied to the system
1/4 1/2 1 푥1 푧1 2 [ 0 −1 −3] [푥2] = [푧2] = [−7] 0 0 21 푥3 푧3 76 and returns the solution x,
푧3 76 푥3 = = ≅ 3.6190 푢33 21 76 −7 − (−3) ( ) 푧2 − 푢23푥3 21 −27 푥2 = = = ≅ −3.8571 푢22 −1 7 1 27 76 2 − ( ) (− ) − (1) ( ) 푧1 − 푢12푥2 − 푢13푥3 2 7 21 26 푥1 = = = − ≅ 1.238 푢11 1 21 4 1.2381 or, x ≅ [−3.8571]. 3.6190
The forward-substitution step can be represented concisely, for a system of 푛 equations, as 푖−1
푧푖 = 푑푖 − ∑ 푙푖푗푧푗 for 푖 = 1,2, … , 푛 푗=1
The back-substitution step can be written concisely as
푧푛 푥푛 = 푢푛푛 푛 푧푖 − ∑푗=푖+1 푢푖푗푥푗 푥푖 = for 푖 = 푛 − 1, 푛 − 2, … ,1 푢푖푖
Matlab’s lu Function and the Left-Division Operator “\”
Matlab has a built-in function “lu” that generates the LU factorization. It has the syntax [L,U,P] = lu(A). This function employs row permutations in order to pivot. These permutations are captured in the matrix P, so that PA=LU.
Example:
Notice how the P matrix permutes (swaps) the first and last rows of matrix A. That happened because the last row has the largest pivot in column one. No swapping of the last two rows occurs because after the first swap, the second row pivot (in column two), 1.0, is larger than the one in the last row, 0.5. The forward and back substitution functions are implemented by the left-division operator “\” (see the next section for more discussion of this operator). The following is an example that illustrates the use of lu and \ to solve a system of linear equations.
Example. Use Matlab to solve the following system of equations using the LU factorization method.
30 −1 −2 푥1 8 [ 1 70 −3 ] [푥2] = [−20] 3 −2 100 푥3 80 LU factorization step:
No row swapping in this case (P=I) because the first row has the largest pivot (in column one), 30, and the second row has the largest pivot (in column two), 70, compared to that in the last row, |-2| = 2.
Forward and back substitution steps:
Alternatively, the substitution steps can be computed in one instruction, as follows (the use of parenthesis is required):
The above example had the permutation matrix 퐏 = 퐈 (the identity matrix). When 퐏 ≠ 퐈, we must take it into account in our formulation, PAx = LUx = Pd. In such case, we must first permute the vector 퐝 by matrix 퐏. The solution is then obtained as, 퐱 = 퐔\[퐋\(퐏퐝)]. The following example illustrates this situation.
Example. Use Matlab to solve the following system of equations using the LU factorization method. 1 1 1 푥1 2 [ 4 2 ] [푥2] = [1] 1 1 1 푥3 8 16 4 1
Since 퐏 ≠ 퐈, the proper way to compute the solution is ,
More on Matlab’s Left-Division Operator
The left-division Matlab operation was used earlier to solve a system of linear equations, Ax = d, using the syntax A\d. Matlab’s left-division is a highly sophisticated algorithm. When used, Matlab examines the structure of the coefficient matrix and then implements an optimal method to obtain the solution. Here is a simplified overview. First, Matlab checks to see if the coefficient matrix (A) is in a format where a solution can be obtained without full Gauss elimination. These include systems that are triangular or symmetric. For example, if A is lower triangular forward- substitution is used to obtain the solution. Similarly, if A is upper triangular back- substitution is used to obtain the solution. If none of the simplified solutions are possible, the Gauss elimination method (with pivoting) is used and the solution is obtained by back-substitution. When the system is near singular (|A| ≅ 0) the user should avoid the use of Matlab’s solution x = inv(A)*d, and should instead employ x = A\d. As an example, let us solve the near singular system [as evidenced by |A| ≅ (6.7)10−16 << 1, or cond(A) ≅ (1.2)1017 >> 1].
1 2 3 푥1 1 [4 5 6] [푥2] = [2] 7 8 9 푥3 3 The use of x = inv(A)*d returns the wrong solution:
(The rcond(A) is the reciprocal of the 1-norm of matrix A. It is computed as 1/cond(A,1). A matrix is ill-conditioned if its rcond number is much smaller than 1. Also note that cond(A)=cond(A,2) is the 2-norm condition number of a matrix. A matrix is ill-conditioned if its cond is much larger than 1.)
Use of x = A\d returns a proper solution (even with an ill-conditioned matrix):
Other solutions do exist for this system {try 퐱 = [−0.0556 0.1111 0.2778]푇 and [−0.233 0.467 0.100]푇}. Left-division is also used to solve over-determined systems of linear equations. An over-determined system has more equations (푚) than unknowns (푛). It arises in regression problems (discussed in later lectures) where it is desirable to fit an analytic function, say a polynomial of 푛 coefficients, to a set of 푚 data points where 푚 > 푛. For such problems, the A matrix is rectangular and generally, there is no solution x that satisfies Ax = d. However, one can cast this problem as an optimization problem, where x is computed so as to minimize the Euclidean norm (or root-square-error), ||Ax − d||, or its squared value (sum-of-squared-error), ||Ax 2 2 푚 2 − d|| ; recall that for an 푚-dimensional vector z, ||z|| =∑푖=1 푧푖 . If we define the column vector d′ = Ax then, the quantity to be minimized is the sum of squared error (SSE) given by 푚 ′ 2 퐼 = ∑(푑푖 − 푑푖) 푖=1 It can be shown (refer to Lecture 18) that the solution to the above optimization problem is the solution to the 푛x푛 system of linear equations (ATA)x = ATd. The solution x is referred to as the least-square-error (LSE) solution. Matlab can compute it as x = (ATA)\(ATd) or as x = pinv(A)*d, where pinv is a built-in Matlab function that is equivalent to the pseudo-inverse of matrix A, given by (ATA)-1AT. The most robust way to solve the over-determined (푚 > 푛) system of equations is to use the left-division operator, x = A\d, just as if it was an 푛x푛 system of equations.
Example. Find the least-squares-error solution for the over-determined system 30 −1 −2 8 푥1 1 70 −3 −20 [ ] [푥2] = [ ] 3 −2 100 80 푥3 4 22 −9 −5 Solution:
The left division-based solution x=A\d can also solve the overdetermined system shown above, as confirmed by the following Matlab session.
The root-square-error, ||Ax − d||, for the solution x is computed as follows (employing Matlab’s norm function):
Example. Let us compare the execution time for Matlab to compute the solution of a 500x500 linear system using left-division and LU factorization-based solutions. The system is assumed to have uniform random elements (over the range [-50 50]) in A and in the five right-hand vectors d1 through d5. The LU factorization method has an advantage since, for the second and higher solutions, only forward and back substitutions are necessary.
So, we see that the LU factorization method is much faster than the Gauss elimination method (as implemented by the \ operator).
Note: We may solve the system subject to all five input vectors using the instruction X = A\D where D = [d1 d2 d3 d4 d5] (a matrix with 5 column vectors). Your turn: Give it a try.
Example. Solve the system of 10x10 linear equations, shown below, that was encountered in Lecture 9 in connection with Pade’ approximation.
−1 0 0 0 0 0 0 0 0 0 푎0 푐0 푐0 0
0 −1 0 0 0 0 푐0 0 0 0 푎1 푐1 푐1 1
0 0 −1 0 0 0 푐 푐 0 0 푎2 푐2 푐2 0 1 0 0 0 0 −1 0 0 푐 푐 푐0 0 푎3 푐3 푐3 −1/3 2 1 푐 푐 푐 푐 푎4 푐 푐 0 0 0 0 0 −1 0 3 2 1 0 4 4 푐 푎 = − 푐 with 푐 = 0 0 0 0 0 −1 푐4 푐3 2 푐1 5 5 5 1/5 푏 0 0 0 0 0 0 푐5 푐4 푐3 푐2 1 푐6 푐6 0 0 0 0 0 0 0 푐6 푐5 푐4 푐3 푏2 푐7 푐7 −1/7 0 0 0 0 0 0 푐7 푐6 푐5 푐4 푏3 푐8 푐8 0 [ 0 0 0 0 0 0 푐8 푐7 푐6 푐5] [푏4] [푐9] [푐9] [ 1/9 ]
Solution: Upon substituting the 푐푖 values we may solve the system of equations employing the left division operator, as follows:
Alternatively, and because of the special nature of the coefficient matrix, the above system can be broken down into two smaller subsystems. The first subsystem is
which can be solved employing any of the techniques discussed so far. Next, we may use the 푏푖 results to solve (by simple substitution) for the 푎푖’s in the following (second) subsystem of equations (obtained from the top six rows of the original matrix formulation),
푎0 = 푐0
푎1 = 푐1 + 푐0푏1
푎2 = 푐2 + 푐1푏1 + 푐0푏2
푎3 = 푐3 + 푐2푏1 + 푐1푏2 + 푐0푏3
푎4 = 푐4 + 푐3푏1 + 푐2푏2 + 푐1푏3 + 푐0푏4
푎5 = 푐5 + 푐4푏1 + 푐3푏2 + 푐2푏3 + 푐1푏4 + 푐0푏5
The following are the numerical results. First, we employ Matlab to compute the 푏푖 coefficients,
So, we obtain
푏1 = 0, 푏2 = 1.1111, 푏3 = 0 and 푏4 = 0.2381
Next, we compute the 푎푖 coefficients employing the above formulas and the values 1 1 푏 = [0 1.1111 0 0.2381] and 푐 = [0 1 0 − 0 ]. 3 5 푎0 = 푐0 = 0
푎1 = 푐1 + 푐0푏1 = 1 + (0)(0) = 1
푎2 = 푐2 + 푐1푏1 + 푐0푏2 = 0 + (1)(0) + (0)(1.1111) = 0 1 푎 = 푐 + 푐 푏 + 푐 푏 + 푐 푏 = − + 0 + (1)(1.1111) + 0 = 0.7778 3 3 2 1 1 2 0 3 3
푎4 = 푐4 + 푐3푏1 + 푐2푏2 + 푐1푏3 + 푐0푏4 = 0 + 0 + 0 + 0 + 0 = 0
푎5 = 푐5 + 푐4푏1 + 푐3푏2 + 푐2푏3 + 푐1푏4 + 푐0푏5 1 1 = + 0 + (− ) (1.1111) + 0 + (1)(0.2381) + 0 = 0.0677 5 3
Iterative Methods: Jacobi and Gauss-Seidel Algorithms
An alternative to elimination methods are iterative methods that are similar in nature to the ones encountered with the fixed-point iterative methods for root finding. The Jacobi iterative method starts with an initial estimate (guess) of the solution vector x(0) and computes a new estimate x(1), using a suitable iterative formulation. Then, it uses x(1) in the iterative formula to compute x(2), and so on. The iterative formulation is derived for the general 3x3 system of linear equations,
푎11푥1 + 푎12푥2 + 푎13푥3 = 푑1
푎21푥1 + 푎22푥2 + 푎23푥3 = 푑2
푎31푥1 + 푎32푥2 + 푎33푥3 = 푑3
Assuming the diagonal elements are non-zero (푎푖푖 ≠ 0) , we can solve the first equation for 푥1, solve the second equation for 푥2 and solve the third equation for 푥3 to obtain (this is along the line of thought as the fixed-point formulation),
푑1 − 푎12푥2 − 푎13푥3 푥1 = 푎11
푑2 − 푎21푥1 − 푎23푥3 푥2 = 푎22
푑3 − 푎31푥1 − 푎32푥2 푥3 = 푎33
The above equations can be represented in matrix form as x = b – Cx, as follows.
푑1 푎 푎 0 12 13 푎 푎 푎 푥 11 11 11 푥 1 푑 푎 푎 1 2 21 23 [푥2] = − 0 [푥2] 푥 푎22 푎22 푎22 푥 3 푎 푎 3 푑3 31 32 0 [푎 푎 ] [ 푎33 ] 33 33 The recursion formula for the Jacobi iteration would then be x(푘 + 1) = b − Cx(푘). Let us apply this method to the following system,
2 −1 1 푥1 2 [1 3 −1] [푥2] = [ 3 ] 8 −4 16 푥3 32
The Jacobi iterative formula for this system can be expressed as
1 1 0 − 1 2 2 1 1 x(푘 + 1)= [1] − 0 − x(푘) 3 3 2 1 1 − 0 [2 4 ]
Now, starting from x(0) = [1 1 1]T, we obtain the following solution estimates from the first two iterations, respectively: 1 1 0 − 1 2 2 1 1 0 1 1 1 x(1) = 퐛 − 퐂퐱(0) = [1] − 0 − [1] = [1] − [0] = [1] 3 3 1 7 2 1 1 1 2 − 0 4 4 [2 4 ] 1 1 3 5 0 − 1 2 2 1 1 8 8 0.625 1 1 1 5 x(2) = 퐛 − 퐂퐱(1) = [1] − 0 − [1] = [1] − − = ≅ [1.250] 3 3 7 4 4 2 1 1 2 1 7 1.75 − 0 4 [2 4 ] [ 4 ] [4] Notice that the estimates of x are converging towards the exact solution, given by
The Jacobi formulation for the b vector and the C matrix for an 푛x푛 system can be computed using the following formulas,
푎 푎 푎 푑1 0 12 13 . 1푛 푎 푎 푎 푎11 11 11 11 푎21 푎23 푎2푛 푑2 0 . 푎22 푎22 푎22 푎22 b = . C = . . 0 ...... 푑푛 푎 푎 푎 푛1 푛2 . 푛(푛−1) 0 [푎푛푛 ] [푎푛푛 푎푛푛 푎푛푛 ]
The convergence of the Jacobi iteration method is much more predictable than for fixed-point iteration of nonlinear equations. It can be shown that convergence is guaranteed if the coefficient matrix A is diagonally dominant, i.e. if 푛
|푎푖푖| ≥ ∑|푎푖푗| for all 푖 = 1, 2 … , 푛 푗=1 푗≠푖 Fortunately, in practice, many engineering problems fulfill this requirement.
In the numerical implementation of the Jacobi method, the iteration is stopped when each and every component 푥푖 meet the absolute relative error criterion, 푥 (푘 + 1) − 푥 (푘) | 푖 푖 | < tol 푥푖(푘 + 1) Alternatively, the iteration stops when the following criterion is met,
푥1(푘 + 1) − 푥1(푘) 푥2(푘 + 1) − 푥2(푘) 푥푛(푘 + 1) − 푥푛(푘) ‖휺푟‖ = ‖ [ … ] ‖ < tol 푥1(푘 + 1) 푥2(푘 + 1) 푥푛(푘 + 1) or, in Matlab syntax, norm((x(푘 + 1) - x(푘)) ./ x(푘 + 1)) < tol where norm is the Matlab norm function that returns the “length” of the vector. The following is a function (Jacobi) that implements the Jacobi iterative method.
Your turn: Analyze the above function so that you gain a good understanding of it. Example. Solve the following system of equations using the Jacobi iterative method starting with initial guess x(0) = 0. Note the diagonal dominance of A.
2 −1 1 푥1 2 [1 3 −1] [푥2] = [ 3 ] 8 −4 16 푥3 32
Example. Solve the following system of equations using the Jacobi iterative method starting with initial guess x(0) = 0. 1 1 1 푥1 2 [ 4 2 ] [푥2] = [1] 1 1 1 푥3 8 16 4 1
The method diverges since the coefficient matrix is not diagonally dominant (note: 100 iterations were performed, by default, before iterations stopped):
The Gauss-Seidel method is very similar to the Jacobi method except that as each new 푥푖 value is computed, it is immediately used in the next formula to determine another 푥푗 value (푗 > 푖). Here, the iterative formulas for a 3x3 system are given by
푑1 − 푎12푥2(푘) − 푎13푥3(푘) 푥1(푘 + 1) = 푎11
푑2 − 푎21푥1(푘 + 1) − 푎23푥3(푘) 푥2(푘 + 1) = 푎22
푑3 − 푎31푥1(푘 + 1) − 푎32푥2(푘 + 1) 푥3(푘 + 1) = 푎33 The following is an illustration of the Gauss-Seidel method (with x(0) = [1 1 1]T) for one of the above examples we used to demonstrate the Jacobi method, namely
2 −1 1 푥1 2 [1 3 −1] [푥2] = [ 3 ] 8 −4 16 푥3 32 Iteration 1:
Iteration 2:
The following table displays the approximations after two iterations using Jacobi and Gauss-Seidel methods along with the exact solution. Notice the faster rate of convergence of the Gauss-Seidel method. (Both iterative methods started with x(0) = [1 1 1 ]T)
Solution Vector Jacobi Method Gauss-Seidel Method Exact Solution (2 iterations) (2 iterations)
푥1 0.625 0.625 0.714
푥2 1.250 1.375 1.429
푥3 1.75 2.031 2.000
True ‖휺푟‖ → 21.7% 13.1% --
The Gauss-Seidel method formulation for a system of 푛 linear equations is given by:
푛 1 푥1(푘 + 1) = [푑1 − ∑ 푎1푗푥푗(푘)] 푎11 푗=2
푖−1 푛 1 푥푖(푘 + 1) = [푑푖 − (∑ 푎푖푗푥푗(푘 + 1) + ∑ 푎푖푗푥푗(푘))] , 푖 = 2,3, … , 푛 − 1 푎푖푖 푗=1 푗=푖+1
푛−1 1 푥푛(푘 + 1) = [푑푛 − ∑ 푎푛푗푥푗(푘 + 1)] 푎푛푛 푗=1
Notice that the values of the unknowns at the 푘 + 1 iteration, 푥푖(푘 + 1), are calculated by using the values 푥푗(푘 + 1) obtained in the 푘 + 1 iteration for 푗 < 푖, and using the values 푥푗(푘) for 푗 > 푖.
The following function (gauss_seidel) implements the Gauss-Seidel method.
Your turn: Analyze the above function so that you gain a good understanding of it. Verification for previous example:
In the above results, the first answer is based on two iterations. The second answer is based on meeting the convergence tolerance of 10−8.
Comparison to the left-division solution method:
In the above results, the Gauss-Seidel solution took 28 iterations to converge and meet the 10−15 tolerance threshold. Note: The left division solution must be used if the A matrix is not diagonally dominant because of divergence problems intrinsic to the iterative Jacobi and Gauss-Seidel methods.