Prof. Ch. Schwab Spring Term 2017 ETH Zurich¨ F. Henriquez D-MATH J. Zech Numerische Mathematik I

Homework Problem Sheet 4

Introduction. Diagonally dominant matrices, LU/Cholesky-decomposition, complexity.

Problem 4.1 LU-decomposition of Strictly Diagonally Dominant Matrices Symmetric positive definite matrices allow stable without pivoting. There are other important classes of matrices that enjoy this property and in this problem you are going to see one. n n×n Definition. A A = (aij)i,j=1 ∈ C is strictly (column) diagonally dominant if for every column of the matrix the magnitude of the diagonal entry is larger than the sum of the magnitudes of all the other (non-diagonal) entries in that column, that is

n X |ajj| > |aij| for all j = 1, . . . , n . (4.1.1) j6=i=1

(4.1a) Show that every symmetric strictly diagonally dominant matrix is regular.

HINT: Consider a kernel vector of A and its component with the largest magnitude.

(4.1b) Show that the Gaussian elimination with partial pivoting (Spaltenpivotsuche) as intro- duced in [NMI, Sect. 2.5] of the lecture notes, of a strictly diagonally dominant matrix always selects the element a11 as pivot element at the first step.

(4.1c) Let A(1) denote the matrix that emerges from the first step of Gaussian elimination of A ∈ Cn×n, that is, after those row transformations that make the subdiagonal entries of the first column vanish.

Show that A(1)(2:n, 2:n) (MATLAB notation for the selection of a sub-matrix) is strictly diago- nally dominant, if A enjoys this property.

HINT: Start with a formula for the entries of A(1).

(4.1d) Show that the Gaussian elimination with partial pivoting of a strictly diagonally domi- nant matrix does not trigger any row swaps.

HINT: Use the result of the two previous subproblems and perform induction with respect to the matrix size.

Problem 4.2 Complexity of Pivoting Strategies (4.2a) Determine the exact number of comparisons in the LU decomposition with column-wise pivoting ([NMI, Alg. 2.28]).

Problem Sheet 4 Page 1 Problem 4.1 (4.2b) Determine the exact number of comparisons in the LU decomposition with complete pivoting (Full Pivoting Remark at the end of [NMI, Ch. 2.5]).

Problem 4.3 Forward and Backward Error of the LU-Decomposition (4.3a) Let A ∈ Rn×n. Prove that if no pivot element in [NMI, Alg. 2.10] is zero, the matrices Lb and Ub of the LU decomposition, computed with the floating-point arithmetic F, satisfy LbUb = A + 4A, |4A| ≤ γn(F)|Lb| |Ub | nu(F) with γn( ) := and where u( ) is the unit roundoff. F 1−nu(F) F

(4.3b) Let A ∈ R2×2 given by 5.3 · 10−4 1 A = . 1 2

Compute the factors Lb and Ub of the LU decomposition of the matrix A in F(10, 4, −10, 10). (4.3c) Using backward–forward substitution compute the solution of the linear system Ax = b in F(10, 4, −10, 10), where −2 b = . 3 Compare with the “exact” solution, i.e. the one obtained without using floating point arithmetic. What can you conclude?

(4.3d) Evaluate the error bound presented in [NMI, Thm. 2.15] for the matrix A from (4.3b).

Problem 4.4 Let 0 6= A ∈ Rn×n be a . (4.4a) Give the definition of positive definiteness for the matrix A.

(4.4b) Show that, if A is positive definite, then aii > 0 for all 1 ≤ i ≤ n. Does the reverse implication hold as well? Justify your answer!

(4.4c) Let A now be positive definite as well. Define the Cholesky-decomposition of A and formulate a sufficient condition that the decomposition can be done.

Problem 4.5 Linear Algebra [Exam Summer 2016] Throughout this problem A ∈ Rn×n, n ∈ N, denotes a symmetric positive definite (SPD) matrix. We wish to compute A−1 in the following way:

i) Compute the Cholesky decomposition R>R of A.

> ii) Solve Axk = R Rxk = ek by forward and backward substitution for k = 1, . . . , n, where ek denotes the k-th unit vector.

−1 iii) Put together A , whose columns consist of the xk.

Problem Sheet 4 Page 2 Problem 4.3 (4.5a) Denote by cost(n) the computational cost of computing A−1 in the above described way (by an efficient implementation). Determine a, b ∈ R such that cost(n) = anb + O(nb−1) as n → ∞.

(4.5b) Does A have an LU-decompositon, and is the LU-decomposition of A unique in this case (no need to give a proof)? Prove that the Cholesky decomposition of A is unique (you do not have to prove existence of the Cholesky decomposition).

(4.5c) Implement:

• [p,R] = cholesky(A): the Cholesky decomposition of a SPD matrix A ∈ Rn×n with full (columns and rows) pivoting. The code must return as output the upper triangular matrix R ∈ Rn×n and the permutation vector p ∈ {1, . . . , n}n (which is a permutation of n > > n×n (i)i=1) such that P AP = R R, where P ∈ R denotes the permutation matrix whose i-th column is the p(i)-th unit vector. In your programme, do not permute the rows and columns of A, but access the entries of A via permuted indices. • x = fbsubs(R,b): forward and backward substitution. The code takes as input the vector b ∈ Rn and the matrix R ∈ Rn×n such that A = R>R is the Cholesky-decomposition of the regular matrix A ∈ Rn×n. Output is the vector x ∈ Rn, solution of the linear system Ax = b. • x = cholsolve(R,p,b): solves the linear system Ax = b for a given SPD matrix A ∈ Rn×n and b ∈ Rn. Here R, p denote the matrix R and the permutation vector belonging to the permutation P, s.t. R>R is the Cholesky decomposition of P>AP (i.e. p, R are the arguments returned by cholesky(A)). Solve the system using fbsubs.m appropriately. To this end, carefully think through how and in which order the respective vectors have to be permuted, and which linear system needs to be solved by fbsubs.m. HINT: The permutation vector q associated to the inverse permutation of p may be com- puted in MATLAB by [∼,q]=sort(p). • B = compinv(A): matrix inversion. Input is the symmetric positive definite matrix A ∈ Rn×n, n ∈ N, and output is B := A−1. Implement the Algorithm as described at the beginning of the problem using cholesky.m (to compute p and R) and cholsolve.m.

Published on March 17, 2017. To be submitted on March 28, 2017. MATLAB: Submit all file in the online system. Include the files that generate the plots. Label all your plots. Include commands to run your functions. Comment on your results.

References

[NMI] Lecture Notes for the course “Numerische Mathematik I”.

Last modified on March 17, 2017

Problem Sheet 4 Page 3 References