MATH 235/W08: Orthogonality & Gram-Schmidt, SOLUTIONS To

Home , Orthogonality

MATH 235/W08: Orthogonality & Gram-Schmidt, SOLUTIONS to Assignment 5

Solutions to questions 3,9,11,13,14.

Contents

1 Orthogonal Pairs 2 1.1 Solution ...... 2

2 Representation using Orthogonal Complement 2 2.1 Solution ...... 2

3 Basis and Distance *** 3 3.1 Solution ...... 3 3.1.1 AHelpfulMATLABProgramwithOutput ...... 4

4 Orthogonal Basis 5

5 Orthogonal Basis for Columns Space 5

6 Orthogonal Matrix 6

7 Orthogonal Matrix Two 6

8 True-False Questions 6

9 Angles Between Vectors *** 6 9.1 Solution ...... 7

10 Orthonormal Bases 7

11 Basis and Nearest Vectors *** 7 11.1Solution ...... 7

12 Gram-Schmidt Procedure 8

13 Gram-Schmidt Procedure and Inner Products *** 8 13.1Solution ...... 9

14 MATLAB: Predator-Prey System *** 10 14.1Solution ...... 10

1 1 Orthogonal Pairs

Determine which pairs of vectors are orthogonal; ﬁnd the norms of each vector. i i a) u = √2, v = √2 1 1   −  1/2 1/√2 1/2  0  b) u = , v = 1/2 √2   −  1/2  0      1.1 Solution

BEGIN SOLUTION: a) T i i T − u, v = u v¯ = √2 √2 =1+2 1 = 0. h i − 6 1 1   −  Therefore, these vectors are not orthogonal. The norms of the two vectors are the same u = v = √1+2+1=2. k k k k b) T 1/2 1/√2 T 1/2 0 1 √2 u, v = u v =     = = 0. h i 1/2 √2 2√2 − 2 6   −  1/2  0      Again, this pair of vectors is not orthogonal. The norms are u 2 = 4/4 = 1 and v 2 = k k k k 1 2= 3 . 2 − − 2

END SOLUTION.

2 Representation using Orthogonal Complement

Let u = (3, 0, 1)T , u = (0, 1, 0)T , and y = (0, 3, 10)T . Let W = span u , u . Represent y as the 1 2 { 1 2} sum of a vector in W and a vector in the orthogonal complement of W .

2.1 Solution

BEGIN SOLUTION: Note that u1, u2 are orthogonal. Therefore, the projection onto W is easily found to be 3 10 3 PW y = u1 + u2 = u1 + 3u2 = 3 . 10 1 1   2 3 3 − And y = Pwy + (y PW y)= 3 +  0  is the required representation. − 1 9 END SOLUTION.    

3 Basis and Distance *** 1 1 3 0 2 − − 0 1 2 1 4 Let be the nullspace of A where A = − and let V 21 0 34   32 1 56 T   v = 2 9 3 32 83 . − − (a) Find a basis for V . (b) Find the distance from v to . V 3.1 Solution

BEGIN SOLUTION: 1 0 1 0 4 − 01 2 0 2 (a) After Gaussian Elimination, we get the RREF  . After a permutation of 00 0 1 2  −  00 0 0 0    I E columns 3 and 4, this matrix is in the form . Therefore, a basis for the nullspace of 0 0 V E A is given by the columns of the matrix (let’s call it V as well) V = , after permuting I − 1 4 −  2 2  the corresponding rows, i.e. V = 1 0 . −   0 2  −   0 1  −  (b) To find the distance from v to , we could find the orthogonal projection of v on , call it P v V V V and let the vector d = v P v. Then the distance is d . This can be done by first finding − V k k an orthogonal basis of , call it S = u , u , and then finding the projection on each of the V { 1 2} two vectors u1, u2. Then P v = Pu1 v + Pu2 v. And the distance is d = v Pu1 v Pu2 v . TV k k k − − k v = 2 9 3 32 83 , and − − proj(v onto v1) = -45/25*v1 proj(v onto v2) = -19/6*v2 ||v - proj(v onto v1) - proj(v onto v2)|| = (7079250/900)^1/2

Alternatively, we could ﬁnd the best vector w for the equation V w = v. This means solving the so-called normal equations. T T ( solve normal equations for w :) V V w = V v.

3 Therefore, the distance from the vector v to the nullspace is V V w v . k − k The calculations are in the MATLAB output below.

END SOLUTION.

3.1.1 A Helpful MATLAB Program with Output A MATLAB program that helps with the above solution is: format compact A=[1 -1 -3 0 2 0 1 2 -1 4 2 1 0 3 4 3 2 1 5 6 ] disp(’find rref(A) using MATLAB - or by pivoting’) find rref(A) using MATLAB - or by pivoting Atemp=rref(A) Atemp = 1 0 -1 0 4 0 1 2 0 2 0 0 0 1 -2 0 0 0 0 0 V=[Atemp(1:3,[3 5]) -eye(2)] V = -1 4 2 2 0 -2 -1 0 0 -1 disp(’columns of V are a basis for the nullspace of A’) columns of V are a basis for the nullspace of A V=V([1 2 4 3 5],:) V = -1 4 2 2 -1 0 0 -2 0 -1 disp(’check that this is indeed a basis of the nullspace’) check that this is indeed a basis of the nullspace A*V ans = 0 0

4 0 0 0 0 0 0 disp(’now find the distance in part b), i.e. ||Vw-v||’) now find the distance in part b), i.e. ||Vw-v|| v=[2 9 -3 32 -83]’ v = 2 9 -3 32 -83 disp(’solve normal equations V^TVw=V^Tv’) solve normal equations V^TVw=V^Tv w=V\v w = 3.1667 1.8000 distvector=V*w-v distvector = 2.0333 0.9333 -0.1667 -35.6000 81.2000 dist=norm(distvector) dist = 88.6895 format short diary off

4 Orthogonal Basis

Find an orthogonal basis for the span of X = (1, 1, 0, 1), X = (0, 1, 1, 1) and X = (1, 0, 1, 1). 1 2 − 3 5 Orthogonal Basis for Columns Space

Find an orthogonal basis for the column space of

1 1 0  1 0 1 A = − . 0 1 1    0 0 1  

5 6 Orthogonal Matrix

2 3 6 Find a constant c such that cA is orthogonal, where A = 6 2 3 has orthogonal columns. − 3 6 2  −  7 Orthogonal Matrix Two

1 1 1 a 2 2 √2  1 1  2 2 0 b Find a, b, c, d such that A = 1 −1 1 is orthogonal.  2 2 √ c  − 2   1 1 0 d  2 − 2  8 True-False Questions

Provide a proof for each true statement and a counter example (or justiﬁcation) for each false statement.

(a) There exist 4 4 orthogonal matrices with rank 3. × (b) If A is a n n orthogonal matrix and B is n 1 matrix, then the equation AX = B has a × × unique solution.

2 T 3 1 T 1 1 T Q (Q ) Q− (Q )− (Q− ) = Q.

(d) The following matrix is orthogonal

1 0 0 1 2 A = 0 1 0 1 1 1 1 . − 3 0 0 1 1    

(e) The matrix A from the previous question satisﬁes A2 A = I. − 9 Angles Between Vectors ***

(a) A vector x = (x ,x ,x ) R3 has the following properties: x = √6, x is orthogonal to 1 2 3 ∈ || || (1, 1, 1), and the angle between x and (1, 1, 0) is θ = π/6. Find the vector (or vectors) x − which satisfy these properties.

(b) Find a non-zero vector in C2 which is orthogonal to both (i, 1) and (1 + i, 2), or show that none exists.

6 9.1 Solution

BEGIN SOLUTION:

x x (a) The orthogonality gives the equation x x +x = 0. The angle condition yields 1+ 2 = √3 1 − 2 3 √6√2 2 or equivalently, x1 + x2 = 3. From

1 1 1 0 1 1 1 0 1 1 1 0 1 0 1/2 3/2 − = − = − = 1 1 03 ∼ 0 2 1 3 ∼ 0 1 1/2 3/2 ∼ 0 1 1/2 3/2 − − − 3 1 2 2 x3 3 − 1 These two equations yield the general solution x =  2 + 2 x3 ,x3 free. To ﬁnd x3 we need x  3  to add the restriction that the norm is √6. and solve the resulting quadratic for x3. This T T yields the two particular solutions: 1 2 1 , 2 1 1 . − (b) The two vectors are linearly independent. Therefore, they span the complete space C2. Therefore, the only vector that is orthogonal to both is 0.

END SOLUTION.

10 Orthonormal Bases

Consider the complex vector space C3 with the standard inner product.

(a) Produce an orthonormal basis for the subspace S spanned by the vectors v1 = (1+ i, 1 2i, 3i) and v2 = (2 + 3i, 1, 1 2i). − − (b) Find the orthogonal projection of v = (2i, 2 i, 1) along v1. − 11 Basis and Nearest Vectors ***

Let W be the subspace of R4 spanned by (1, 0, 1, 2) and (2, 1, 0, 1) and take the inner product − − to be the standard dot product.

(a) Find the vector in W which is closest to the vector (4, 3, 2, 1).

(b) Find a basis for W ⊥ and the dimension of W ⊥.

BEGIN SOLUTION:

7 (a) Let u = (1, 0, 1, 2) and u = (2, 1, 0, 1). These two vectors are orthogonal. Let v = 1 − 2 − (4, 3, 2, 1). Therefore, we can take the projection to ﬁnd the closest vector, call it w1,

T T (v u1) (v u2) 2 2 w1 = u1 u1 + u2 u2 k k 1 k k 2 24 0 1 10 = (4/6)   + (10/6)   = 1   . 1 0 6 4 −    −   2   1  2   −  −  (b) Since W is the span of two orthogonal, nonzero vectors, the dimension of W is 2, i.e. dim W = 4 2. Since W is a subspace of R , the dimension of W ⊥ is also 2. We can ﬁnd a basis for W ⊥ 1 0 1 2 by ﬁnding the nullspace of the matrix − . One can easily observe that the 2 1 0 1 − 1 0 −  2 1 columns of − form a basis for the nullspace and so a basis for W . 1 2 ⊥ − −   0 1  − 

12 0 T 1 5 1 4 w = v w = (4, 3, 2, 1)   =   . 2 − 1 − 3 2 3 8 −     1 4 −   

END SOLUTION.

12 Gram-Schmidt Procedure

Let x1 = (1, 1, 1, 1), x2 = (1, 0, 0, 1), x3 = (1, 1, 3, 3) in R4 and let S = Span(x1, x2, x3). − − (a) If the Gram-Schmidt procedure is applied to the vectors x1, x2, x3 in that order, ﬁnd the resulting orthonormal basis for S.

(b) Find a basis for S⊥. (c) Find an orthonormal basis for R4 which extends the orthonormal basis for S found in part (a).

13 Gram-Schmidt Procedure and Inner Products ***

Using the Gram-Schmidt procedure and the standard inner product in R4, and in C4:

1. Find an orthonormal basis for the span of the three vectors

T T T 1 01 0 , 1 0 1 0 , 034 0 . { − }

8 2. construct an orthonormal set of vectors from the set: 1 i 0 0  i   0   1   0  x1 = , x2 = , x3 = , x4 = . 0 2 0 0          0   0   1   i          13.1 Solution

BEGIN SOLUTION: First note that MATLAB has NND5GS Gram-Schmidt demonstration.

1. Denote the three vectors: T T T y := 1 01 0 ,y := 1 0 1 0 ,y := 03 40 . 1 2 − 3 Set v1 = y1. Now subtracting oﬀ the projection of y2 on v1 shows that they were orthogonal. So, we can choose v = y . Now subtracting oﬀ the projection of y on the span of v , v 2 2 3 { 1 2} yields 0 1 1 0 T T y3 v1 y3 v2 3 0  0  3 v3 = y3 v1 v2 = 2 + 2 = − vT v − vT v 4 − 1 1 0 1 1 2 2     −    0 0  0  0         We can now normalize the three orthogonal vectors to get the orthonormal basis

1 1 0  1 0 1 0 1    ,   ,   . √2 1 √2 1 0     −    0  0  0          2. There are many ways of doing this. But, for simplicity, let’s start with x4 since it is already 0 0 normalized, i.e. we set v = ix =   = e . We can now continue with x . Applying 1 − 4 0 4 3   1   0 1 1 0 the GS step yields v =   = e . We can now use x to get v =   = e . And ﬁnally 2 0 2 1 3 0 1     0 0     using x2 yields the ﬁnal vector e3. Therefore, the basis we get this way is the standard basis of unit vectors.

END SOLUTION.

9 14 MATLAB: Predator-Prey System ***

(MATLAB) Linear Recurrences: A Predator-Prey System Linear recurrence equations are well-suited to studying predator-prey models in the biolog- ical sciences. The following example involves studying the dynamics between two inter-related populations–hawks and mice. Let hk and mk denote the number of hawks and mice, respectively, in a certain area in year k. Assume that they are related by the following equations:

hk+1 = 0.5hk + 0.01mk mk = 12.5hk + 1.25mk +1 − for k 0. ≥ hk If we write Sk = , then these equations become mk

Sk+1 = ASk

Find matrix A corresponding to the above. Find also the diagonalizing matrix P and the diagonal matrix D for A using MATLAB. 1 k k 1 k 1 Recall that A = P DP − . This means that A = P D P − and hence that Sk = P D P − S1, where S1 is the column vector whose entries are the initial populations of hawks and mice. Let the initial populations consist of 40 hawks and 1500 mice. It follows that the limiting population vector SL is given by

SL = lim Sk k →∞ k 1 = lim P D P − S k 1 →∞ k 1 = P ( lim D )P − S k 1 →∞

Use this formula and MATLAB to calculate the limiting population

hL SL = mL

What is the limiting population of hawks, hL? What is the limiting population of mice, mL?

14.1 Solution

BEGIN SOLUTION: The solution can be found using the eigenvalue decomposition, i.e. note that the eigenvalues are .75, 1 and the eigenvector corresponding to the larger eigenvalue 1 is (.02, 1). We get the limiting population 1000(.02, 1), for hawks and mice. A MATLAB program is: clear all !rm temp.txt

10 diary temp.txt A= [ .5 .01; -12.5 1.25] [P,D]=eig(A) S0=[40;1500] Snew=S0; niter=90; % change this number as desired for i=1:niter, Sold=Snew; Snew=A*Sold; disp(num2str(Snew’)) end disp([’using MATLAB and ’,num2str(niter),’ iterations, the limiting population is’]) [Snew] diary off

END SOLUTION.