The Projection Matrix David Arnold Fall 1996 Abstract In this activity you will use Matlab to project a set of vectors onto a single vector. Prerequisites. Inner product (dot product) and orthogonal vectors. 1 The Inner Product We begin with the de nition of the inner product. De nition 1 Let K and L be vectors from 4n. The inner product (dot prod- uct) of vectors K and L is de ned by K ¢ L = KT L · ¸ · ¸ 1 ¡4 Example 2 If K = and L = , then 7 2 T K ¢ L = K L · ¸ £ ¤ ¡4 = 17 2 =10 Check this result in Matlab. >> u=[1;7] u= 1 7 >> v=[-4;2] v= -4 1 2 >> u*v ans = 10 If K, L and M are vectors from 4n and c is any real number, then it is not di¢cult to show that each of the following properties are true: ² K ¢ L = L ¢ K ² K ¢ (L + M) = K ¢ L + K ¢ M ² (cK) ¢ L = K ¢ (cL) = c(K ¢ L) ² K ¢ K ¸ 0 and K ¢ K =0if and only if K = . Finally, orthogonal (perpendicular) vectors are de ned as follows. De nition 3 Let K and L be vectors from 4n. Vectors K and L are orthogonal if and only if K ¢ L =0. 1.1 The Projection of One Vector Onto Another Figure 1 shows the projection of vector K onto vector L. u projLu v Figure 1. The projection of K onto L. In Figure 2, it is clear that the projection of K onto L is some scalar multiple of L; that is, projLK = cL. 2 u u-cv cv v Figure 2. The vector K ¡ cL is orthogonal to L. Note that the di¤erence vector K ¡ cL is orthogonal (perpendicular) to L. Consequently, L ¢ (K ¡ cL) =0 and we can use the previously mentioned properties to arrive at the following result: L ¢ K ¡ L¢(cL) =0 L ¢ K ¡ c(L ¢ L) =0 c(L ¢ L) = L ¢ K L ¢ K c = (1) L ¢ L Finally, substitute result (1) into projLK = cL. L ¢ K proj K = L (2) L L ¢ L · ¸ 1 Example 4 Use formula (2) to nd the projection of K = onto L = 7 · ¸ ¡4 : 2 L ¢ K projLK = L L·¢ L ¸ · ¸ ¡4 1 ¢ · ¸ 2 7 ¡4 = · ¸ · ¸ ¡4 ¡4 2 ¢ 2 2 · ¸ 10 ¡4 = 20 2 · ¸ ¡2 = 1 Check this result in Matlab. 3 >> (v*u)/(v*v)*v ans = -2 1 1.2 The Projection Matrix Lets rework formula (2). First, when you multiply a vector by a scalar, it doesnt matter whether you position the scalar before or after the vector. L ¢ K proj K = L L L ¢ L L ¢ K = L L ¢ L Next, use the transpose de nition of the inner product followed by the associa- tive property of multiplication. Remember, when performing the dot product, a scalar multiplier may be placed anywhere you wish. 1 proj K = L(LT K) L LT L 1 = (LLT )K LT L LLT = K LT L The expression LLT is called an outer product (the transpose operator is outside the product versus its inside position in the inner product). If we de ne LLT P = , then the projection formula becomes LT L LLT proj K = PK, where P = : L LT L The matrix P is called the projection matrix. You can project any vector onto the vector L by multiplying by the matrix P . · ¸ · ¸ 1 ¡4 Example 5 Let K = onto L = and nd P, the matrix that will 7 2 project any matrix onto the vector L. Use the result to nd projLK. First, nd the projection matrix. LLT P = LT L · ¸· ¸ ¡4 ¡4 T 2 2 = · ¸ · ¸ ¡4 T ¡4 2 2 4 · ¸ ¡4 £ ¤ ¡42 2 = · ¸ £ ¤ ¡4 ¡42 2 · ¸ 16 ¡8 ¡84 = · 20 ¸ : 8 ¡: 4 = ¡: 4 : 2 Check this result in Matlab. >> P=(v*v)/(v*v) P= 0.8000 -0.4000 -0.4000 0.2000 Use this result to nd the projection of K onto L. proj K = PK L · ¸· ¸ : 8 ¡: 4 1 = ¡: 4 : 2 7 · ¸ ¡2 = 1 Check this last result in Matlab. >> P*u ans = -2 1 Note that this is identical to the result in Example 4. 1.3 Projecting a Lot of Vectors onto a Single Vector First, use Matlab to generate 100 random vectors and plot them. The following commands will produce an image similar to that in Figure 3. >> U=8*rand(2,100)-4; >> x=U(1,:); >> y=U(2,:); >> plot(x,y,o) 5 4 2 0 -2 -4 -4 -2 0 2 4 Figure 3. The image in Figure 3 bears some explanation. Each circle in Figure 3 represents the tip of a vector whose tail begins at the origin. Next, use the projection matrix P to project each of the 100 2 £1 vectors in matrix U onto the vector L, then plot the results on the graph in the color red. >> PU=P*U; >> x=PU(1,:); >> y=PU(2,:); >> hold on >> plot(x,y,ro) 4 2 0 -2 -4 -4 -2 0 2 4 Figure 4. In Figuure 4, note that each vector· in¸ the matrix U is projected onto a line ¡4 in the direction of the vector L = . 2 2 Homework 1. Create 100 random 2 £ 1 vectors in a matrix U with components between ¡2 and 2. Plot them. 6 (a) Create a matrix P which will· project¸ each of the 2 £ 1 vectors in 1 matrix U onto the vector L = . 1 (b) Use the matrix P to project each 2 £ 1 vector in matrix U onto the vector L. Plot the results in a second color. (c) Obtain a printout of your result. 2. The column space of the projection matrix P is a line in the direction of the vector L. (a) What is the dimension of the column space of P ? Check your answer with the ATLAST distributions colbasis command. (b) What is the rank of P ? Check your answer with Matlabs rank command. 3. If matrix P projects all vectors onto the vector L, what should the matrix PP do? Can you explain geometrically why the matrices PP and P are equal? Check this in Matlab. 7.