Math 552 Scientific Computing II Spring 2010

SOLUTIONS: Homework Set 2

1. Write a MATLAB m-file (function) called mylusolve.m that takes the output a and p from your mylu.m m-file, along with a righthand side b, and computes the solution of Ax = b by performing the forward and backward substitution steps. The first line of your code mylusolve.m should be as follows:

function x = mylusolve(a,p,b) (your code here!)

Execute the following commands and turn in a copy of mylusolve.txt:

>> diary mylusolve.txt >> type mylusolve.m >> a = rand(4); b = sum(a,2); >> [a,p] = mylu(a); >> x = mylusolve(a,p,b) >> norm(x-[1 1 1 1]’,’inf’) >> diary off

ANS: Here is mylusolve.txt:

type mylusolve.m

function x = mylusolve(a,p,b) % n = length(p); x = zeros(n,1); z = zeros(n,1);

% forward sub to solve Lz=b z(1) = b(p(1)); for i = 2:n s = 0; for j = 1:i-1 s = s+a(p(i),j)*z(j); end z(i) = b(p(i))-s; end

% backsub to solve Ux=z x(n) = z(n)/a(p(n),n); for i = n-1:-1:1 s = 0; for j = i+1:n s = s+a(p(i),j)*x(j); end x(i) = (z(i)-s)/a(p(i),i); end a = rand(4); b = sum(a,2); [a,p] = mylu(a); x = mylusolve(a,p,b) x =

1.0000 1.0000 1.0000 1.0000 norm(x-[1 1 1 1]’,’inf’) ans =

8.8818e-16 diary off 2. The famous Hilbert matrices are given by Hij = 1/(i + j − 1). The n × n Hilbert matrix Hn is easily produced in MATLAB using hilb(n). Assume the true solution of Hnx = b T for a given n is x = [1,..., 1] . Hence the righthand side b is simply the row sums of Hn, and b is easily computed in MATLAB using b=sum(hilb(n)’)’. Use your codes mylu.m and mylusolve.m to solve the system Hnx = b for n = 5, 10, 15, 20. For each n, using the ∞−norm, compute the relative error and the relative residual. Discuss what is happening here. You may find it useful to look at the cond command in MATLAB. ANS: Here is a code to do the computations and the output:

n = [5 10 15 20]’; rel_err = zeros(4,1); rel_resid = zeros(4,1); cond_H = zeros(4,1);

for i = 1:length(n) H = hilb(n(i)); % generate matrix b = sum(H,2); % sum rows for RHS b [H,p] = mylu(H); % solve system x = mylusolve(H,p,b); rel_err(i) = max(abs(ones(n(i),1)-x)); % note ||x||_inf = 1 H = hilb(n(i)); % we need the original H rel_resid(i) = max(abs(b-H*x))/max(abs(b)); cond_H(i) = cond(H,’inf’); end

disp(’ n rel_err rel_resid cond ’) disp(’ ------’) disp([n rel_err rel_resid cond_H]);

Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 9.144066e-19. Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND = 1.291654e-19.

n rel_err rel_resid cond ------5.0000e+00 5.5467e-13 1.9449e-16 9.4366e+05 1.0000e+01 1.3982e-04 1.5162e-16 3.5353e+13 1.5000e+01 2.4345e+01 4.0150e-16 1.0975e+18 2.0000e+01 4.7309e+01 3.7031e-16 9.2117e+18

So we see that while for each n the relative residual is on the order of machine precision, the relative error is growing. This is directly attributable to the growth in the condition number of the Hilbert matrices. In fact, I’ve included the warnings that MATLAB displayed for −1 n = 15, 20 to note that it is having trouble accurately computing Hn . 3. This probem emphasizes the difficulty of computing interpolating polynomials in standard form. The matrices which arise are called Vandermonde matrices, and can be quite ill- conditioned. Consider the polynomial of degree n that interpolates a set of data {Fi} at the points {xi} in the form

n X i Pn(x) = aix i=0 and has the property that Pn(xi) = Fi.

(a) Write down the matrix system (for a general n) that must be solved to compute the ai’s. (b) Compute the coefficients for the polynomial that interpolates the function

n X F (x) = xi i=0

i at the n + 1 points xi = 1 + n for i = 0, . . . , n (i.e. Fi = F (xi)). Do this for n = 7, 9, 11, 13. The MATLAB code for n = 7 is >> format long e >> n=7; >> x=(1+(0:1/n:1))’; >> V=fliplr(vander(x)); >> F = sum(V,2); >> a = V\F; You should use the help in MATLAB to make sure you understand what each of these commands. Note that fliplr is used so that V corresponds to the the matrix from part (a), as well as the definition of V given in class last term. (c) Compute the ∞-norm condition number of each Vandermode matrix V in part (b). Use MATLAB’s cond command for this.

(d) Compute the relative error in the ∞-norm of the computed coefficients ai. The true th answer is given by the n degree polynomial with ai = 1 for i = 0, . . . , n. Discuss the results in lieu of the condition numbers from (c).

ANS: (a) For each i we have an equation, and the (n + 1) × (n + 1) system V a = F that one must solve is given by

 2 n      1 x0 x0 ··· x0 a0 F0 2 n  1 x1 x ··· x   a1   F1   1 1       . . . .   a2   F2   . . . .    =   .  2 n   .   .   1 xn−1 xn−1 ··· xn−1   .   .  2 n 1 xn xn ··· xn an Fn

For parts (b)-(d) here is a MATLAB code and ouput: CODE: n = [7 9 11 13]’; rel_err = zeros(4,1); cond_V = zeros(4,1); format long e for i = 1:length(n) x = (1+(0:1/n(i):1))’; % interpolation nodes V = fliplr(vander(x));; % generate Vandermonde matrix cond_V(i) = cond(V,’inf’); % compute condition number F = sum(V,2); % sum rows for RHS b a = V\F; % solve system disp([’n = ’, num2str(n(i))]), disp([’a = ’]), disp(a) disp(’ ’) rel_err(i) = max(abs(ones(n(i)+1,1)-a)); % note ||a_true||_inf = 1 end disp(’ n rel_err cond ’) disp(’ ------’) disp([n rel_err cond_V]);

OUTPUT: n = 7 a = 9.999999995647673e-01 1.000000002074155e+00 9.999999958136149e-01 1.000000004640408e+00 9.999999969480911e-01 1.000000001191393e+00 9.999999997442813e-01 1.000000000023290e+00 n = 9 a = 1.000000163540988e+00 9.999989784432664e-01 1.000002815820528e+00 9.999955041820527e-01 1.000004582815625e+00 9.999969066011437e-01 1.000001382893095e+00 9.999996051174539e-01 1.000000065365618e+00 9.999999952202283e-01 n = 11 a = 1.000021007657227e+00 9.998395822077357e-01 1.000553854312332e+00 9.988586628504313e-01 1.001559913058271e+00 9.985151562201965e-01 1.001004538974869e+00 9.995169442902384e-01 1.000161821998705e+00 9.999640301683905e-01 1.000004775098920e+00 9.999997131626831e-01 n = 13 a = 9.914092903599439e-01 1.078480422008933e+00 6.704572440779317e-01 1.842170357231789e+00 -4.614436788171800e-01 2.818571685511569e+00 -6.695045903943608e-01 2.144875334185679e+00 4.135212097342278e-01 1.221659184732196e+00 9.399191271078082e-01 1.011059923579160e+00 9.987606385808471e-01 1.000063852101455e+00

n rel_err cond ------7.000000000000000e+00 4.640407613720754e-09 8.980828455113095e+08 9.000000000000000e+00 4.582815625431280e-06 5.035429847646414e+11 1.100000000000000e+01 1.559913058270634e-03 3.061787776747201e+14 1.300000000000000e+01 1.818571685511569e+00 1.938482966085804e+17

So for part (b), we see as n gets larger the computed solution begins to diverge from the exact solution, which is a vector of all ones. For part (c), looking at the output just above, in particular the infinity-norm condition number of the corresponding Vandermonde matrix, we can see that it grows rapidly, and is already O(1017) for n = 13, indicating that these systems are indeed highly ill-conditioned. For (d), looking again at the table above, we see that the difference in the magnitude of the relative error and condition number is about, again, O(1017). That is, there is a direct correlation, as expected, in the relative error and the condition number. When the latter is large we generally start to lose accuracy in the computed solution. 4. Consider the matrix, right side vector, and two approximate solutions,

 1.2969 0.8648   0.8642   0   0.9911  A = , b = , x = , x = . 0.2161 0.1441 0.1440 1 1 2 −0.4870

(a) Show that x = [2, −2]T is the exact solution of Ax = b.

(b) Compute the error and residual vectors for x1 and x2. −1 (c) Use MATLAB to find ||A||∞, ||A ||∞, and κ∞(A). (d) In class we proved a theorem relating the condition number of A, the relative error, and the relative residual. Check this result for the two approximate solutions x1 and x2 (using the ∞ − norm). ANS: For (a), a direct calculation gives

 1.2969 0.8648   2   2 ∗ 1.2969 − 2 ∗ 0.8648   0.8642  Ax = = = = b 0.2161 0.1441 −2 2 ∗ 0.2161 − 2 ∗ 0.1441 0.1440

Using MATLAB, for (b) we have

>> A = [1.2969 0.8648; 0.2161 0.1441]; b = [0.8642 0.1440]’; >> x1 = [0 1]’; x2 = [0.9911 -0.4870]’; x = [2 2]’; >> e1 = x-x1, r1 = b-A*x1

e1 =

2 1

r1 =

-0.000600000000000045 -0.000100000000000017

>> e2 = x-x2, r2 = b-A*x2

e2 =

1.0089 2.487

r2 =

1.00000001612699e-08 -9.99999999473644e-09 For (c), again using MATLAB

>> A_inf = norm(A,’inf’), Ainv_inf = norm(inv(A),’inf’), A_condinf = cond(A,’inf’)

A_inf =

2.1617

Ainv_inf =

151300000.022015

A_condinf =

327065210.047589 So while the condition number of A is small, the condition number of A−1 is quite large, resulting in kAkkA−1k = κ(A) ≈ 3.27 × 108. For (d), we proved in class that for any norm and corresponding induced matrix norm kek krk krk ≤ κ(A) = kAkkA−1k . kxk kbk kbk To see that this holds here in the infinity norm,

>> norm(e1,’inf’)/norm(x,’inf’) , A_condinf*norm(r1,’inf’)/norm(b,’inf’) ans =

1 ans =

227076.054187188

>> norm(e2,’inf’)/norm(x,’inf’) , A_condinf*norm(r2,’inf’)/norm(b,’inf’) ans =

1.2435 ans =

3.78460096415374 which indeed it does. Note that the relative residual for x1 is much larger than that for x2, while the error in x2 is larger. 5. (Matrix norms and Spectral radius)

(a) Consider the matrix,  2 −3 1  A =  −4 1 2  . 5 0 1

Compute kAk∞ and find a vector x such that kAk∞ = kAxk∞/kxk∞.

(b) Find an example of a 2 × 2 matrix A such that kAk∞ = 1 but ρ(A) = 0. Explain why this shows that the spectral radius ρ(A) is not a matrix norm.

(c) Given an n×n matrix A, prove that the 1-norm of A, kAk1, equals the maximum absolute column sum of A, namely:

n X kAk1 = max kAxk1 = max |aij|. kxk =1 1≤j≤n 1 i=1

ANS:

(a) Since kAk∞ is the maximum absolute column sum of A, it is easy to see that kAk∞ = T T 11. Letting x = [1 −1 1] , we see that kxk∞ = 1 and kAxk∞/kxk∞ = k[2 4 5] k∞/1 = 11.  0 0  (b) Let A = . Then it is easily seen that kAk = 1 but ρ(A) = 0. This follows 1 0 ∞ since the infinity norm of A is just the maximum absolute column sum, and for a lower triangular matrix the eigenvalues lie on the diagonal. Since they are both 0, the spectral radius is 0. This show that ρ(A) is not a matrix norm, for we have found a non-zero matrix for which ρ(A) = 0. This violates one of the properties that any matrix norm must satisfy, namely, kAk = 0 ⇐⇒ A = 0. (c) For any n-vector x 6= 0 we have

Pn Pn kAxk1 = i=1 | j=1 aijxj| Pn Pn ≤ i=1 j=1 |aij||xj| Pn Pn = j=1 |xj| i=1 |aij| Pn ≤ (max1≤j≤n i=1 |aij|) kxk1

Dividing by kxk1 6= 0 shows that

n ! kAxk1 X max ≤ max |aij| . x6=0 kxk 1≤j≤n 1 i=1 To show that there is some x 6= 0 for which equality is achieved, let k be such that

n n X X max |aij| = |aik|, 1≤j≤n i=1 i=1

then if ek is the k-th unit vector and ak the k-th colum of A,

n X kAekk1 = kakk1 = |aik| = kAk1, i=1 and we have thus proven that

n X kAk1 = max |aij| = maximum absolute row sum of A. 1≤j≤n i=1