DOCSLIB.ORG

A Fast Converging Distributed Solver for Linear Systems with Generalised Diagonal Dominance

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Fast Converging Distributed Solver for Linear Systems with Generalised Diagonal Dominance 1 A Fast Converging Distributed Solver for Linear Systems with Generalised Diagonal Dominance Qianqian Cai1, Zhaorong Zhang2 and Minyue Fu2;1, Fellow, IEEE Abstract—This paper proposes a new distributed algorithm for In this paper, we study distributed solutions for sparse solving linear systems associated with a sparse graph under a linear systems. Our focus is on linear systems which sat- generalised diagonal dominance assumption. The algorithm runs isfy the so-called generalised diagonal dominance condition. iteratively on each node of the graph, with low complexities on local information exchange between neighbouring nodes, local These systems include those with diagonal dominance as a computation and local storage. For an acyclic graph under the subset (those with a diagonally dominant matrix). Diagonally condition of diagonal dominance, the algorithm is shown to dominant matrices have vast applications in engineering, eco- converge to the correct solution in a finite number of iterations, nomics and finance [1], [2]. For example, many matrices equalling the diameter of the graph. For a loopy graph, the that arise in finite element modeling are diagonally dominant algorithm is shown to converge to the correct solution asymptot- ically. Simulations verify that the proposed algorithm significantly [32]; diagonally dominant matrices also arise naturally for outperforms the classical Jacobi method and a recent distributed covariance of partially correlated random vectors [33]. The linear system solver based on average consensus and orthogonal generalised diagonal dominance condition relaxes the diagonal projection. dominance requirement using a diagonal scaling matrix, i.e., Index Terms—Distributed algorithm; distributed optimization; only a diagonally scaled matrix is required to be diagonally distributed estimation; linear systems. dominant. As we will point out later, the generalised diagonal dominance condition is also equivalent to the so-called walk- summability condition [12], [34]. I. INTRODUCTION We propose a new distributed algorithm for solving sparse Sparse linear systems are of great interest in many disci- linear systems with generalised diagonal dominance. The algo- plines, and many iterative methods exist for solving sparse rithm runs iteratively on each node of the graph associated with linear systems; see, e.g., [1], [2], [3], [4], [5], [6], [7], [8], the system, and enjoys low complexities on local information [9], [10], [11], [12]. However, for many networked system exchange between neighboring nodes, local computation and applications, distributed algorithms are preferred. The main local storage. For an acyclic graph, the algorithm converges difference is that an iterative method focuses on reducing the to the exact solution in d iterations, where d is the diameter total computation by incorporating the sparsity of the system, of the graph. This is also the theoretical minimum number of but allowing a central controller for the coordination of the iterations required to solve a linear system using a distributed algorithm. In contrast, a distributed algorithm is designed algorithm. For a loopy graph, the algorithm is guaranteed to such that the same algorithm is executed concurrently on converge to the exact solution asymptotically. every node (or sub-system, or agent) while allowing only We have compared the proposed algorithm with the classical neighbouring nodes to share information. Ideally, no central Jacobi method [1], [2], [3] and the distributed linear system controller or coordinator is required, and each node should solver [16] based on average consensus and orthogonal projec- have low complexities in terms of information exchange, tion to show that our algorithm has much faster convergence. computation and storage. For example, a sequential iterative The development of the algorithm and its theoretical anal- algorithm, where the nodes are ordered and their executions ysis follows from a similar algorithm known as Gaussian belief propagation (BP) algorithm [35] which applies to Gauss- arXiv:1904.12313v1 [eess.SP] 28 Apr 2019 are done sequentially, should not be regarded as a distributed algorithm. For a large networked system such as sensor Markov random fields which involve a symmetric system ma- networks [13], [14], [15], networked control systems [16], trix; see, e.g., [10], [12]. Namely, the convergence properties in [17], [18], network-based state estimation [19], [20], [21], [10] are for symmetric matrices with diagonal dominance, and [22], biological networks [23], [24], multi-agent systems [25], the convergence properties in [12] are for symmetric matrices [26], [27], [28], multi-agent optimization [29], [30], [31] and with generalised diagonal dominance (or walk summability). so on, it is often desirable to use distributed algorithms rather The main contribution of our work in comparison with those than iterative methods. in [10], [12] can be viewed as generalising the Gaussian BP algorithm to solving linear systems with an asymmetric system 1School of Automation, Guangdong University of Technology, and Guang- matrix. dong Key Laboratory of IoT Information Processing, Guangzhou 510006, China. 2School of Electrical Engineering and Computer Science, The University II. PROBLEM FORMULATION AND PRELIMINARIES of Newcastle. University Drive, Callaghan, 2308, NSW, Australia. This work was supported by the National Natural Science A. Problem Formulation Foundation of China (Grant Nos. 61633014 and U1701264). E- mails: [email protected]; [email protected]; Consider a network of nodes (1; 2; : : : ; n) associated with n [email protected]. a state vector x = colfx1; x2; : : : ; xng 2 R . The information 2 available at each node i is that x satisfies a linear system: C3: Local storage: Each node i’s storage should be at most O(jNij) over all iterations. aT x = b ; i i Lemma 1: Under the local information exchange constraint n where ai = colfai1; ai2; : : : aing 2 R is a column vector above, a connected graph G with diameter d needs a minimum and bi is a scalar. Collectively, the common state satisfies of d iterations to solve the distributed linear system problem with a general A and b. Ax = b (1) Proof: Let nodes i and j be such that they are d hops T T T away from each other (such nodes exist by the definition of with A = colfa1 ; a2 ; : : : ; an g and b = colfb1; b2; : : : ; bng. It is assumed throughout the paper that (i) A is invertible; and (ii) d). The correct solution for xi requires the information of all the bj connected to node i, which needs to propagate to node aii 6= 0 for all i = 1; 2; : : : ; n. Note that (ii) can be guaranteed under (i) because nonzero diagonals can always be obtained i. By the local information exchange constraint, this will take through proper pivoting (swapping of rows of columns) [2]. at least d iterations. Denote the solution of (1) by Remark 1: Although we focus only on linear systems with square and invertible matrix A, under-determined and x? = A−1b: (2) over-determined linear systems can be treated with slight modifications. For the under-determined case (where the row Define the induced graph G = fV; Eg with V = rank of A is less than n), we can zero-pad (A; b) to make A f1; 2; : : : ; ng and E = f(i; j)jaij 6= 0 or aji 6= 0g. We square, then solve the regularised problem of (A + λI)x = b associate each node i with (aii; bi), and each edge (i; j) with for some sufficiently small scalar or diagonal matrix λ > 0. aij. Note in particular that G is undirected, i.e., (i; j) 2 E For the over-determined case (where A has full column rank if and only if (j; i) 2 E. For each node i 2 V, we and has more rows than columns), it is standard to solve the define its neighbouring set as Ni = fjj(i; j) 2 Eg and T least-squares problem minx(Ax − b) (Ax − b) instead, which denote its cardinality by jNij. We assume in the sequel results in the linear system (AT A)x = (AT b) with a square that jNij n for all i 2 V. The graph G is said to be and invertible (AT A). connected if for any i; j 2 V, there exists a (connecting) Remark 2: The well-celebrated Jacobi method (see, e.g., [1], path of (i; i1); (i1; i2);:::; (ik; j) 2 E. The distance between [2], [3]) for solving a linear system with diagonal dominance two nodes is the minimum number of edges connecting the is an excellent example of distributed algorithms. Denoting by two nodes. It is obvious that a graph with a finite number ? x^i(k) the estimate of the i-th component of x , this method of nodes is either connected or composed of a finite number is simply rewritten as of disjoint subgraphs with each of them being a connected 0 1 graph. The diameter of a connected graph is defined to be 1 X largest distance between two nodes in the graph. The diameter x^i(k + 1) = @bi − aijx^j(k)A (3) aii of a disconnected graph is the largest diameter of a connected j2Ni subgraph. By this definition, the diameter of a graph with a with the initial condition of xi(0) = bi=aii, i = 1; 2; : : : ; n. finite number of nodes (or a finite graph) is always finite. A It is straightforward to verify that constraints C1)-C3) are loop is defined to be a path starting and ending at node i all met by this method. In contrast, consider the well-known through a node j 6= i. A graph is said to be acyclic if it does enhanced version of the Jacobi method known as the Gauss- not contain any loop. A cyclic (or loopy) graph is a graph with Seidel algorithm (also see [1], [2], [3]): at least one loop. 0 1 The distributed linear system problem we are interested in 1 X X is to devise an iterative algorithm for every node i 2 V to x^i(k + 1) = Bbi − aijx^j(k + 1) − aijx^j(k)C (4) aii@ A execute so that the computed estimate x^i(k) at iteration j<i j>i ? j2Ni j2Ni k = 0; 1; 2;:::; will approach xi as k increases.
Load more

Recommended publications
  • The Invertible Matrix Theorem
    The Invertible Matrix Theorem Ryan C. Daileda Trinity University Linear Algebra Daileda The Invertible Matrix Theorem Introduction It is important to recognize when a square matrix is invertible. We can now characterize invertibility in terms of every one of the concepts we have now encountered. We will continue to develop criteria for invertibility, adding them to our list as we go. The invertibility of a matrix is also related to the invertibility of linear transformations, which we discuss below. Daileda The Invertible Matrix Theorem Theorem 1 (The Invertible Matrix Theorem) For a square (n × n) matrix A, TFAE: a. A is invertible. b. A has a pivot in each row/column. RREF c. A −−−→ I. d. The equation Ax = 0 only has the solution x = 0. e. The columns of A are linearly independent. f. Null A = {0}. g. A has a left inverse (BA = In for some B). h. The transformation x 7→ Ax is one to one. i. The equation Ax = b has a (unique) solution for any b. j. Col A = Rn. k. A has a right inverse (AC = In for some C). l. The transformation x 7→ Ax is onto. m. AT is invertible. Daileda The Invertible Matrix Theorem Inverse Transforms Definition A linear transformation T : Rn → Rn (also called an endomorphism of Rn) is called invertible iff it is both one-to-one and onto. If [T ] is the standard matrix for T , then we know T is given by x 7→ [T ]x. The Invertible Matrix Theorem tells us that this transformation is invertible iff [T ] is invertible.
    [Show full text]
  • Math 54. Selected Solutions for Week 8 Section 6.1 (Page 282) 22. Let U = U1 U2 U3 . Explain Why U · U ≥ 0. Wh
    Math 54. Selected Solutions for Week 8 Section 6.1 (Page 282) 2 3 u1 22. Let ~u = 4 u2 5 . Explain why ~u · ~u ≥ 0 . When is ~u · ~u = 0 ? u3 2 2 2 We have ~u · ~u = u1 + u2 + u3 , which is ≥ 0 because it is a sum of squares (all of which are ≥ 0 ). It is zero if and only if ~u = ~0 . Indeed, if ~u = ~0 then ~u · ~u = 0 , as 2 can be seen directly from the formula. Conversely, if ~u · ~u = 0 then all the terms ui must be zero, so each ui must be zero. This implies ~u = ~0 . 2 5 3 26. Let ~u = 4 −6 5 , and let W be the set of all ~x in R3 such that ~u · ~x = 0 . What 7 theorem in Chapter 4 can be used to show that W is a subspace of R3 ? Describe W in geometric language. The condition ~u · ~x = 0 is equivalent to ~x 2 Nul ~uT , and this is a subspace of R3 by Theorem 2 on page 187. Geometrically, it is the plane perpendicular to ~u and passing through the origin. 30. Let W be a subspace of Rn , and let W ? be the set of all vectors orthogonal to W . Show that W ? is a subspace of Rn using the following steps. (a). Take ~z 2 W ? , and let ~u represent any element of W . Then ~z · ~u = 0 . Take any scalar c and show that c~z is orthogonal to ~u . (Since ~u was an arbitrary element of W , this will show that c~z is in W ? .) ? (b).
    [Show full text]
  • The Invertible Matrix Theorem II in Section 2.8, We Gave a List of Characterizations of Invertible Matrices (Theorem 2.8.1)
    i i “main” 2007/2/16 page 312 i i 312 CHAPTER 4 Vector Spaces For Problems 9–12, determine the solution set to Ax = b, 14. Show that a 6 × 4 matrix A with nullity(A) = 0 must and show that all solutions are of the form (4.9.3). have rowspace(A) = R4. Is colspace(A) = R4? 13−1 4 15. Prove that if rowspace(A) = nullspace(A), then A 9. A = 27 9 , b = 11 . contains an even number of columns. 15 21 10 16. Show that a 5×7 matrix A must have 2 ≤ nullity(A) ≤ 2 −114 5 7. Give an example of a 5 × 7 matrix A with 10. A = 1 −123 , b = 6 . nullity(A) = 2 and an example of a 5 × 7 matrix 1 −255 13 A with nullity(A) = 7. 11−2 −3 17. Show that 3 × 8 matrix A must have 5 ≤ nullity(A) ≤ 3 −1 −7 2 8. Give an example of a 3 × 8 matrix A with 11. A = , b = . 111 0 nullity(A) = 5 and an example of a 3 × 8 matrix 22−4 −6 A with nullity(A) = 8. 11−15 0 18. Prove that if A and B are n × n matrices and A is 12. A = 02−17 , b = 0 . invertible, then 42−313 0 nullity(AB) = nullity(B). 13. Show that a 3 × 7 matrix A with nullity(A) = 4 must have colspace(A) = R3. Is rowspace(A) = R3? [Hint: Bx = 0 if and only if ABx = 0.] 4.10 The Invertible Matrix Theorem II In Section 2.8, we gave a list of characterizations of invertible matrices (Theorem 2.8.1).
    [Show full text]
  • Rotation Matrix - Wikipedia, the Free Encyclopedia Page 1 of 22
    Rotation matrix - Wikipedia, the free encyclopedia Page 1 of 22 Rotation matrix From Wikipedia, the free encyclopedia In linear algebra, a rotation matrix is a matrix that is used to perform a rotation in Euclidean space. For example the matrix rotates points in the xy -Cartesian plane counterclockwise through an angle θ about the origin of the Cartesian coordinate system. To perform the rotation, the position of each point must be represented by a column vector v, containing the coordinates of the point. A rotated vector is obtained by using the matrix multiplication Rv (see below for details). In two and three dimensions, rotation matrices are among the simplest algebraic descriptions of rotations, and are used extensively for computations in geometry, physics, and computer graphics. Though most applications involve rotations in two or three dimensions, rotation matrices can be defined for n-dimensional space. Rotation matrices are always square, with real entries. Algebraically, a rotation matrix in n-dimensions is a n × n special orthogonal matrix, i.e. an orthogonal matrix whose determinant is 1: . The set of all rotation matrices forms a group, known as the rotation group or the special orthogonal group. It is a subset of the orthogonal group, which includes reflections and consists of all orthogonal matrices with determinant 1 or -1, and of the special linear group, which includes all volume-preserving transformations and consists of matrices with determinant 1. Contents 1 Rotations in two dimensions 1.1 Non-standard orientation
    [Show full text]
  • On Bounds and Closed Form Expressions for Capacities Of
    On Bounds and Closed Form Expressions for Capacities of Discrete Memoryless Channels with Invertible Positive Matrices Thuan Nguyen Thinh Nguyen School of Electrical and School of Electrical and Computer Engineering Computer Engineering Oregon State University Oregon State University Corvallis, OR, 97331 Corvallis, 97331 Email: [email protected] Email: [email protected] Abstract—While capacities of discrete memoryless channels are That said, it is still beneficial to find the channel capacity well studied, it is still not possible to obtain a closed form expres- in closed form expression for a number of reasons. These sion for the capacity of an arbitrary discrete memoryless channel. include (1) formulas can often provide a good intuition about This paper describes an elementary technique based on Karush- Kuhn-Tucker (KKT) conditions to obtain (1) a good upper bound the relationship between the capacity and different channel of a discrete memoryless channel having an invertible positive parameters, (2) formulas offer a faster way to determine the channel matrix and (2) a closed form expression for the capacity capacity than that of algorithms, and (3) formulas are useful if the channel matrix satisfies certain conditions related to its for analytical derivations where closed form expression of the singular value and its Gershgorin’s disk. capacity is needed in the intermediate steps. To that end, our Index Terms—Wireless Communication, Convex Optimization, Channel Capacity, Mutual Information. paper describes an elementary technique based on the theory of convex optimization, to find closed form expressions for (1) a new upper bound on capacities of discrete memoryless I. INTRODUCTION channels with positive invertible channel matrix and (2) the Discrete memoryless channels (DMC) play a critical role optimality conditions of the channel matrix such that the upper in the early development of information theory and its ap- bound is precisely the capacity.
    [Show full text]
  • Chapter 9 Eigenvectors and Eigenvalues
    Chapter 9 Eigenvectors and Eigenvalues 9.1 Eigenvectors and Eigenvalues of a Linear Map Given a finite-dimensional vector space E,letf : E E ! be any linear map. If, by luck, there is a basis (e1,...,en) of E with respect to which f is represented by a diagonal matrix λ1 0 ... 0 0 λ ... D = 2 , 0 . ... ... 0 1 B 0 ... 0 λ C B nC @ A then the action of f on E is very simple; in every “direc- tion” ei,wehave f(ei)=λiei. 511 512 CHAPTER 9. EIGENVECTORS AND EIGENVALUES We can think of f as a transformation that stretches or shrinks space along the direction e1,...,en (at least if E is a real vector space). In terms of matrices, the above property translates into the fact that there is an invertible matrix P and a di- agonal matrix D such that a matrix A can be factored as 1 A = PDP− . When this happens, we say that f (or A)isdiagonaliz- able,theλisarecalledtheeigenvalues of f,andtheeis are eigenvectors of f. For example, we will see that every symmetric matrix can be diagonalized. 9.1. EIGENVECTORS AND EIGENVALUES OF A LINEAR MAP 513 Unfortunately, not every matrix can be diagonalized. For example, the matrix 11 A = 1 01 ✓ ◆ can’t be diagonalized. Sometimes, a matrix fails to be diagonalizable because its eigenvalues do not belong to the field of coefficients, such as 0 1 A = , 2 10− ✓ ◆ whose eigenvalues are i. ± This is not a serious problem because A2 can be diago- nalized over the complex numbers.
    [Show full text]
  • The Invertible Matrix Theorem
    Math 240 TA: Shuyi Weng Winter 2017 February 6, 2017 The Invertible Matrix Theorem Theorem 1. Let A 2 Rn×n. Then the following statements are equivalent. 1. A is invertible. 2. A is row equivalent to In. 3. A has n pivots in its reduced echelon form. 4. The matrix equation Ax = 0 has only the trivial solution. 5. The columns of A are linearly independent. 6. The linear transformation T defined by T (x) = Ax is one-to-one. 7. The equation Ax = b has at least one solution for every b 2 Rn. 8. The columns of A span Rn. 9. The linear transformation T defined by T (x) = Ax is onto. 10. There exists an n × n matrix B such that AB = In. 11. There exists an n × n matrix C such that CA = In. 12. AT is invertible. Theorem 2. Let T : Rn ! Rn be defined by T (x) = Ax. Then T is one-to-one and onto if and only if A is an invertible matrix. Problem. True or false (all matrices are assumed to be n × n, unless otherwise specified). 1. The identity matrix is invertible. 2. If A can be row reduced to the identity matrix, then it is invertible. 3. If both A and B are invertible, so is AB. 4. If A is invertible, then the matrix equation Ax = b is consistent for every b 2 Rn. n 5. If A is an n × n matrix such that the equation Ax = ei is consistent for each ei 2 R a column of the n × n identity matrix, then A is invertible.
    [Show full text]
  • Block Matrices in Linear Algebra
    PRIMUS Problems, Resources, and Issues in Mathematics Undergraduate Studies ISSN: 1051-1970 (Print) 1935-4053 (Online) Journal homepage: https://www.tandfonline.com/loi/upri20 Block Matrices in Linear Algebra Stephan Ramon Garcia & Roger A. Horn To cite this article: Stephan Ramon Garcia & Roger A. Horn (2020) Block Matrices in Linear Algebra, PRIMUS, 30:3, 285-306, DOI: 10.1080/10511970.2019.1567214 To link to this article: https://doi.org/10.1080/10511970.2019.1567214 Accepted author version posted online: 05 Feb 2019. Published online: 13 May 2019. Submit your article to this journal Article views: 86 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=upri20 PRIMUS, 30(3): 285–306, 2020 Copyright # Taylor & Francis Group, LLC ISSN: 1051-1970 print / 1935-4053 online DOI: 10.1080/10511970.2019.1567214 Block Matrices in Linear Algebra Stephan Ramon Garcia and Roger A. Horn Abstract: Linear algebra is best done with block matrices. As evidence in sup- port of this thesis, we present numerous examples suitable for classroom presentation. Keywords: Matrix, matrix multiplication, block matrix, Kronecker product, rank, eigenvalues 1. INTRODUCTION This paper is addressed to instructors of a first course in linear algebra, who need not be specialists in the field. We aim to convince the reader that linear algebra is best done with block matrices. In particular, flexible thinking about the process of matrix multiplication can reveal concise proofs of important theorems and expose new results. Viewing linear algebra from a block-matrix perspective gives an instructor access to use- ful techniques, exercises, and examples.
    [Show full text]
  • Lecture 12 Elementary Matrices and Finding Inverses
    Mathematics 13: Lecture 12 Elementary Matrices and Finding Inverses Dan Sloughter Furman University January 30, 2008 Dan Sloughter (Furman University) Mathematics 13: Lecture 12 January 30, 2008 1 / 24 I A is invertible. I The only solution to A~x = ~0 is the trivial solution ~x = ~0. I The reduced row-echelon form of A is In. ~ ~ n I A~x = b has a solution for b in R . I There exists an n × n matrix C for which AC = In. Result from previous lecture I The following statements are equivalent for an n × n matrix A: Dan Sloughter (Furman University) Mathematics 13: Lecture 12 January 30, 2008 2 / 24 I The only solution to A~x = ~0 is the trivial solution ~x = ~0. I The reduced row-echelon form of A is In. ~ ~ n I A~x = b has a solution for b in R . I There exists an n × n matrix C for which AC = In. Result from previous lecture I The following statements are equivalent for an n × n matrix A: I A is invertible. Dan Sloughter (Furman University) Mathematics 13: Lecture 12 January 30, 2008 2 / 24 I The reduced row-echelon form of A is In. ~ ~ n I A~x = b has a solution for b in R . I There exists an n × n matrix C for which AC = In. Result from previous lecture I The following statements are equivalent for an n × n matrix A: I A is invertible. I The only solution to A~x = ~0 is the trivial solution ~x = ~0. Dan Sloughter (Furman University) Mathematics 13: Lecture 12 January 30, 2008 2 / 24 ~ ~ n I A~x = b has a solution for b in R .
    [Show full text]
  • Similar Matrices and Jordan Form
    Similar matrices and Jordan form We’ve nearly covered the entire heart of linear algebra – once we’ve finished singular value decompositions we’ll have seen all the most central topics. AT A is positive definite A matrix is positive definite if xT Ax > 0 for all x 6= 0. This is a very important class of matrices; positive definite matrices appear in the form of AT A when computing least squares solutions. In many situations, a rectangular matrix is multiplied by its transpose to get a square matrix. Given a symmetric positive definite matrix A, is its inverse also symmet­ ric and positive definite? Yes, because if the (positive) eigenvalues of A are −1 l1, l2, · · · ld then the eigenvalues 1/l1, 1/l2, · · · 1/ld of A are also positive. If A and B are positive definite, is A + B positive definite? We don’t know much about the eigenvalues of A + B, but we can use the property xT Ax > 0 and xT Bx > 0 to show that xT(A + B)x > 0 for x 6= 0 and so A + B is also positive definite. Now suppose A is a rectangular (m by n) matrix. A is almost certainly not symmetric, but AT A is square and symmetric. Is AT A positive definite? We’d rather not try to find the eigenvalues or the pivots of this matrix, so we ask when xT AT Ax is positive. Simplifying xT AT Ax is just a matter of moving parentheses: xT (AT A)x = (Ax)T (Ax) = jAxj2 ≥ 0.
    [Show full text]
  • MATH. 513. JORDAN FORM Let A1,...,Ak Be Square Matrices of Size
    MATH. 513. JORDAN FORM Let A1,...,Ak be square matrices of size n1, . , nk, respectively with entries in a field F . We define the matrix A1 ⊕ ... ⊕ Ak of size n = n1 + ... + nk as the block matrix A1 0 0 ... 0 0 A2 0 ... 0 . . 0 ...... 0 Ak It is called the direct sum of the matrices A1,...,Ak. A matrix of the form λ 1 0 ... 0 0 λ 1 ... 0 . . 0 . λ 1 0 ...... 0 λ is called a Jordan block. If k is its size, it is denoted by Jk(λ). A direct sum J = Jk1 ⊕ ... ⊕ Jkr (λr) of Jordan blocks is called a Jordan matrix. Theorem. Let T : V → V be a linear operator in a finite-dimensional vector space over a field F . Assume that the characteristic polynomial of T is a product of linear polynimials. Then there exists a basis E in V such that [T ]E is a Jordan matrix. Corollary. Let A ∈ Mn(F ). Assume that its characteristic polynomial is a product of linear polynomials. Then there exists a Jordan matrix J and an invertible matrix C such that A = CJC−1. Notice that the Jordan matrix J (which is called a Jordan form of A) is not defined uniquely. For example, we can permute its Jordan blocks. Otherwise the matrix J is defined uniquely (see Problem 7). On the other hand, there are many choices for C. We have seen this already in the diagonalization process. What is good about it? We have, as in the case when A is diagonalizable, AN = CJ N C−1.
    [Show full text]
  • Matrix Inverses Recall
    Matrix inverses Recall... DefinitionMatrix inversesA square matrix A is invertible (or nonsingular) if matrix 9 BRecall...such that AB = I and BA = I. (We say B is an inverse of A.) Definition A square matrix A is invertible (or nonsingular) if matrix B Remark Not all square matrices are invertible. ∃ such that AB = I and BA = I. (We say B is an inverse of A.) Theorem.Remark IfNotA allis squareinvertible, matrices then are its invertible. inverse is unique. 1 RemarkTheorem.WhenIf A Ais invertible,is invertible, then we its denote inverse itsis unique. inverse as A− . 1 Remark When A is invertible, we denote its inverse as A− . Theorem. If A is an n n invertible matrix, then the system of × ~ 1~ linearTheorem. equationsIf A given is an n by A~xn invertible= b has matrix, the unique then solutionthe system~x of= linearA− b. × 1 equations given by Ax = b has the unique solution x = A− b. Proof. Assume A is an invertible matrix. Then we have by associativity of by def'n of Proof. Assume A is an invertiblematrix m matrix.ult. Theniden wetit havey 1 1 A(A− b)=(AA− )b = Ib = b. by def'n of inverse Theorem (Properties1 of matrix inverse). Thus, ~x = A− ~b is a solution to A~x = ~b. 1 1 1 Suppose(a) If A~y isis another invertible, solution then A to− theis itself linear invertible system. and It follows(A− )− that= AA~y. = ~b, 1 1 but multiplying both sides by A− gives ~y = A− ~b = ~x. (b) If A is invertible and c =0is a scalar, then cA is invertible and 1 1 1 (cA)− = c A− .
    [Show full text]