Material science Paper No. : Mathematical tools for materials Module : Indirect methods for linear equations Development Team Prof. Vinay Gupta, Department of Physics and Astrophysics, Principal Investigator University of Delhi, Delhi Prof. P. N. Kotru ,Department of Physics, University of Jammu, Paper Coordinator Jammu-180006 Content Writer Prof. V. K. Gupta , Department of Physics, University of Delhi, Delhi-110007 Prof Mahavir Singh Department of Physics, Himachal Pradesh Content Reviewer University, Shimla 1 Crystallography & crystal growth Material science Experimental methods for x-ray diffraction Description of Module Subject Name Physics Paper Name Mathematical tools for physics Module Name/Title Indirect methods for linear Equations Module Id NA-2 2 Crystallography & crystal growth Material science Experimental methods for x-ray diffraction TABLE OF CONTENTS 1. Introduction 2. Vector Norms 2.1 Distance between vectors in ℝ푛 2.2 Convergent sequences 3. Matrix norms and distances 4. The Jacobi iterative method 4.1 The matrix form 5. The Gauss-Seidel method 5.1 The matrix form 6. Convergence of iterative techniques 6.1Criterion for convergence 3 Crystallography & crystal growth Material science Experimental methods for x-ray diffraction LEARNING OBJECTIVES 1 Various vector norms are introduced. The associated notion of vector distance is also introduced. 2 Idea of convergence is defined and limit of a sequence of vectors introduced. 3 Next, matrix norms and distances. 4 The Jacobi iterative method is described and also put in the matrix form. 5 Next the Gauss-Seidel method is described. This is also put in the matrix form as well. 6 Convergence of iterative techniques is discussed and simple criterion for convergence described. 4 Crystallography & crystal growth Material science Experimental methods for x-ray diffraction Indirect methods for solution of linear Equations 1. Introduction In this unit we describe iterative techniques for solving linear systems. Jacobi and the Gauss-Seidel iterative methods are the two classic iterative methods in this class. Iterative techniques are seldom used for solving linear systems of small dimension since the time required for sufficient accuracy exceeds that required for direct techniques such as Gauss elimination. For large systems with a high percentage of 0 entries, however, these techniques are often much more efficient. Systems of this type arise frequently in circuit analysis and in the numerical solution of boundary-value problems and partial-differential equations. 2. Vector norms Before we begin to describe these methods, we introduce certain theoretical ideas about norms of matrices and vectors which are useful in the analysis of the Jacobi and the Gauss-Seidel iterative techniques. We already introduced the idea of a vector norm earlier. Now we will introduce two types of vector norms, the l2 and l∞ T norms of a vector x={x1, x2, …..xn} [the vector x is a column; it is being written as the transpose of a row only to save space.] The l2 and l∞ norms of a vector are respectively 푛 2 1/2 ‖퐱‖2 = {∑푖=1 푥푖 } (1) ‖퐱‖∞ = max |푥푖| (2) 푖=1,푛 The l2 norm is what we called the Euclidean norm of the vector x. It is not difficult to show that the norms defined above do satisfy the properties required of a norm that we described in the unit on matrices. Example 푇 The l2 and l∞ norms of the vector 퐱 = {1,2, −2} are respectively 푛 2 1/2 ‖퐱‖2 = {∑푖=1 푥푖 } = 3 ‖퐱‖∞ = max |푥푖| = 2 푖=1,푛 2.1 Distance between vectors in ℝ푛 The distance between two vectors is defined as the norm of the difference of the vectors. Thus 푛 2 1/2 ‖퐱 − 퐲‖2 = {∑푖=1(푥푖 − 푦푖) } (3) ‖퐱 − 퐲‖∞ = max |푥푖 − 푦푖| (4) 푖=1,푛 Example Let 퐱 = {1,3,5}푇; 퐲 = {3,2,3}푇. Then 푛 2 1/2 ‖퐱 − 퐲‖2 = {∑푖=1(푥푖 − 푦푖) } = 3 ‖퐱 − 퐲‖∞ = max |푥푖 − 푦푖| = 2 푖=1,푛 2.2 Convergent sequence 5 Crystallography & crystal growth Material science Experimental methods for x-ray diffraction The concept of distance in ℝ푛 is also used to define a limit of a sequence of vectors in this space. A sequence of 푘 ∞ 푛 vectors {퐱 }푘=1 in ℝ is said to converge to x with respect to the norm ‖. ‖ if, given any ε > 0, there exists an integer N(ε) such that ‖퐱(푘) − 퐱‖ < 휀 for all 푘 ≥ 푁(휀) Regarding the convergence of sequences of vectors, we have the following useful theorems. Theorem-1 푘 ∞ 푛 The sequence of vectors {퐱 }푘=1 converges to x in ℝ with respect to the l∞ norm if, and only if, 푘 lim 푥 = 푥푖; 푖 = 1,2, ⋯ 푛 푘→∞ 푖 Proof (k) Suppose {x } converges to x with respect to the l∞ norm. Then, given any ε > 0, there exists an integer n(ε) such that for all k > n(ε) (푘) (푘) max |푥 − 푥푖| = ‖퐱 − 퐱‖ < 휀 푖=1 푡표 푛 푖 ∞ (푘) 푘 This implies that |푥 − 푥푖| < 휀 ⟹ lim (푥푖) = 푥푖 for each 푖 푖 푘→∞ The converse can be proved in a similar manner. Example 2 3 Consider the sequence of vectors {1, 2 + , 3 + , 4푒−푘}푇. For 푘 → ∞ it converges to {1,2,3,0}T with respect to 푘 푘2 the l∞ norm. Further, given any ε > 0, there exists an integer N(ε/2) with the property that 휀 휖 ‖퐱(푘) − 퐱‖ ≤ whenever 푘 ≥ 푁( ) ∞ 2 2 By the theorem-1 above, this implies that 2휖 ‖퐱(퐤) − 퐱‖ ≤ √4‖퐱(푘) − 퐱‖ ≤ = 휀 2 ∞ 2 휖 (k) when 푘 ≥ 푁 ( ). So x converges to x with respect to l2 norm as well. 2 Theorem-2 For each 푛 푥 ∈ ℝ , ‖퐱‖∞ ≤ ‖퐱‖2 ≤ √푛‖퐱‖∞. Proof Let j be such that |xj| is the maximum of |xi| for i = 1, 2, ...n. Then 2 2 2 푛 2 2 ‖퐱‖∞ = |푥푗| = 푥푗 ≤ ∑푖=1 푥푖 = ‖퐱‖2 Or ‖퐱‖∞ ≤ ‖퐱‖2 6 Crystallography & crystal growth Material science Experimental methods for x-ray diffraction Further, since xj is the maximum in magnitude 2 푛 2 푛 2 2 2 ‖퐱‖2 = ∑푖=1 푥푖 ≤ ∑푖=1 푥푗 = 푛푥푗 = 푛‖퐱‖∞ ⟹ ‖퐱‖2 ≤ √푛‖퐱‖∞ Combining the two inequalities we get the desired result ‖퐱‖∞ ≤ ‖퐱‖2 ≤ √푛‖퐱‖∞. QED 3. Matrix norms and distances A matrix norm on the set of all n × n matrices is a real-valued function, ‖. ‖ defined on this set, satisfying the following properties for all n × n matrices A and B and all real numbers α: (i) ‖퐀‖ ≥ 0, ‖퐀‖ = 0 if, and only if, the matrix 퐀 is a null matrix. (ii) ‖α퐀‖ = |훼|‖퐀‖ (iii) ‖퐀 + 퐁‖ ≤ ‖퐀‖ + ‖퐁‖ (iv) ‖퐀퐁‖ ≤ ‖퐀‖‖퐁‖ The distance between n × n matrices A and B with respect to this matrix norm is ‖퐀 − 퐁‖. Although matrix norms can be obtained in various ways, the norms that we consider are those that follow from the vector norms l2 and l∞. These norms are defined using the following theorem: Theorem-3 For every vector norm ‖. ‖ ‖퐀‖ = max ‖퐀퐱‖ (5) ‖퐱‖=1 is a matrix norm. Thus every vector norm defines a matrix norm. In addition there could be, and are, matrix norms which have no counterpart in vector norms. A matrix norm defined by a vector norm is called the natural or induced matrix norms associated with that vector norm. If 퐱 ≠ 0, the vector 퐳 = 퐱/|퐱| is a unit vector. Hence 퐱 ‖퐀퐱‖ max‖퐀퐳‖ = max ‖퐀( )‖ = max ‖퐳‖=1 ‖퐱‖≠0 |퐱| ‖퐱‖≠0 ‖퐱‖ We can therefore alternately write ‖퐀퐱‖ ‖퐀‖ = max (6) ‖퐱‖≠0 ‖퐱‖ From this result follows the following corollary: for any vector 퐱 ≠ 0, any A, and any natural matrix norm ‖퐀퐱‖ ≤ ‖퐀‖‖퐱‖ (7) From the l2 and l∞ vector norms, there follow the l∞ matrix norm ‖퐀‖∞ = max ‖퐀퐱‖∞ (8) ‖퐱‖∞=1 and the l2 matrix norm 7 Crystallography & crystal growth Material science Experimental methods for x-ray diffraction ‖퐀‖2 = max ‖퐀퐱‖2 (9) ‖퐱‖2=1 Theorem-4 The l∞ norm of a matrix can be easily computed by the use of the following theorem: If A = (aij) is an n x n matrix, 푛 ‖퐀‖∞ = max ∑푗=1 |푎푖푗|. (10) 푖=1,2,⋯푛 Proof Let x be an n-dimensional vector with ‖퐱‖∞ = max |푥푖| = 1. Since Ax is also a vector, 푖=1,⋯푛 푛 푛 푛 ‖퐴퐱‖∞ = max |(퐀퐱)푖| = max |∑푗=1 푎푖푗푥푗| ≤ max ∑푗=1 |푎푖푗| max |∑푗=1 푥푗| 푖=1,⋯푛 푖=1,⋯푛 푖=1,⋯푛 푗=1,⋯푛 푛 But max |∑푗=1 푥푗| = ‖퐱‖∞ = 1, so that 푗=1,⋯푛 ‖퐀퐱‖∞ ≤ max ‖퐀퐱‖∞ ‖퐱‖∞=1 And consequently ‖퐀‖∞ = ‖퐀퐱‖∞ ≤ max ‖퐀퐱‖∞ ‖퐱‖∞=1 Let l be an integer such that 푛 ∑푗=1 |푎푙푗| = max |푎푖푗| 푖=1,⋯푛 and x be a vector with components 1 푖푓 푎푙푗 ≥ 0 푥푗 = { −1 푖푓 푎푙푗 < 0 Then ⟦퐱⟧∞ = 1; 푎푙푗푥푗 = |푎푙푗|; 푗 = 1,2, ⋯ 푛, and this implies 푛 푛 푛 푛 ‖퐴푥‖∞ = max |∑푗=1 푎푖푗푥푗| ≥ |∑푗=1 푎푙푗푥푗| = |∑푗=1 푎푙푗| = max |∑푗=1 푎푖푗| 푖=1,⋯푛 푖=1,⋯푛 Or 푛 ‖퐀‖∞ = ‖퐀퐱‖∞ ≥ max |∑푗=1 푎푖푗| 푖=1,⋯푛 On combining the two inequalities, we get 푛 ‖퐀‖∞ = max |∑푗=1 푎푖푗| QED 푖=1,⋯푛 Example 1 2 −1 For the matrix 퐴 = [2 −1 2 ], 3 −2 −2 3 ∑푗=1 |푎1푗| = |1| + |2| + |−1| = 4 8 Crystallography & crystal growth Material science Experimental methods for x-ray diffraction 3 ∑푗=1 |푎2푗| = |2| + |−1| + |2| = 5 3 ∑푗=1 |푎3푗| = |3| + |−2| + |−2| = 7 Hence ‖퐀‖∞ = max{4,5,7} = 7 Theorem-5 The l2 norm of a matrix is related to the spectral radius of the matrix.