Lecture 23: 6.1 Inner Products Wei-Ta Chu 2008/12/17 Definition An inner product on a real vector space V is a function that associates a real number u, vwith each pair of vectors u and v in V in such a way that the following axioms are satisfied for all vectors u, v, and w in V and all scalars k. u, v= v, u u + v, w= u, w+ v, w ku, v= k u, v u, u0 and u, u=0if andonlyif u = 0 A real vector space with an inner product is called a real inner product space. 2008/12/17 Elementary Linear Algebra 2 Euclidean Inner Product on Rn If u = (u1, u2, …, un) and v = (v1, v2, …, vn) are vectors in Rn, then the formula v, u= u · v = u1v1 + u2v2 + … + unvn defines v, uto be the Euclidean product on Rn. The four inner product axioms hold by Theorem 4.1.2. 2008/12/17 Elementary Linear Algebra 3 Properties of Euclidean Inner Product Theorem 4.1.2 n If u, v and w are vectors in R and k is any scalar, then u · v = v · u (u + v) · w = u · w + v · w (k u) · v = k (u · v) v · v ≥0; Further, v · v = 0 if and only if v = 0 Example (3u + 2v) · (4u + v) = (3u) · (4u + v) + (2v) · (4u + v ) = (3u) · (4u) + (3u) · v + (2v) · (4u) + (2v) · v =12(u · u) + 11(u · v) + 2(v · v) 2008/12/17 Elementary Linear Algebra 4 Euclidean Inner Product vs. Inner Product The Euclidean inner product is the most important inner product on Rn. However, there are various applications in which it is desirable to modify the Euclidean inner product by weighting its terms differently. More precisely, if w1,w2,…,wn are positive real numbers, and if u=(u1,u2,…,un) and v=(v1,v2,…,vn) are vectors in Rn, then it can be shown which is called weighted Euclidean inner product with weights w1,w2,…,wn … 2008/12/17 Elementary Linear Algebra 5 Example Suppose that some physical experiment can produce any of n possible numerical values x1,x2,…,xn. We perform m repetitions of the experiment and yield these values with various frequencies; that is, x1 occurs f1 times, x2 occurs f2 times, and so forth. f1+f2+…+fn=m. Thus, the arithmetic average, or mean, is 2008/12/17 Elementary Linear Algebra 6 Example If we let f=(f1,f2,…,fn), x=(x1,x2,…,xn), w1=w2=…=wn=1/m Then this equation can be expressed as the weighted inner product 2008/12/17 Elementary Linear Algebra 7 Weighted Euclidean Product 2 Let u = (u1, u2) and v = (v1, v2) be vectors in R . Verify that the weighted Euclidean inner product u, v= 3u1v1 + 2u2v2 satisfies the four product axioms. Solution: Note first that if u and v are interchanged in this equation, the right side remains the same. Therefore, u, v= v, u. If w = (w1, w2), then u + v, w= 3(u1+v1)w1 + 2(u2+v2)w2 = (3u1w1 + 2u2w2) + (3v1w1 + 2v2w2)= u, w+ v, w which establishes the second axiom. 2008/12/17 Elementary Linear Algebra 8 Weighted Euclidean Product ku, v=3(ku1)v1 + 2(ku2)v2 = k(3u1v1 + 2u2v2) = k u, v which establishes the third axiom. 2 2 2 v, v= 3v1v1+2v2v2 = 3v1 + 2v2 .Obviously , v, v= 3v1 + 2 2 2 2v2 ≥0 . Furthermore, v, v= 3v1 + 2v2 = 0 if and only if v1 = v2 = 0, That is , if and only if v = (v1,v2)=0. Thus, the fourth axiom is satisfied. 2008/12/17 Elementary Linear Algebra 9 Definition If V is an inner product space, then the norm (or length) of a vector u in V is denoted by ||u|| and is defined by ||u|| = u, u½ The distance between two points (vectors) u and v is denoted by d(u,v) and is defined by d(u, v) = ||u –v|| If a vector has norm 1, then we say that it is a unit vector. 2008/12/17 Elementary Linear Algebra 10 Norm and Distance in Rn If u = (u1, u2, …, un) and v = (v1, v2, …, vn) are vectors in Rn with the Euclidean inner product, then d(,),()()uv uv uvuv 1/ 2 uvuv 1/ 2 2 2 2 (u1 v 1 ) ( u 2 v 2 ) ... ( un v n ) 2008/12/17 Elementary Linear Algebra 11 Weighted Euclidean Inner Product The norm and distance depend on the inner product used. If the inner product is changed, then the norms and distances between vectors also change. 2 For example, for the vectors u = (1,0) and v = (0,1) in R with the Euclidean inner product, we have u 1 and d(u, v) u v (1,1) 12 (1)2 2 However, if we change to the weighted Euclidean inner product u, v= 3u1v1 + 2u2v2 , then we obtain u u,u 1/ 2 (311200)1/ 2 3 d(u, v) u v (1,1),(1,1) 1/ 2 [3112(1)(1)]1/ 2 5 2008/12/17 Elementary Linear Algebra 12 Unit Circles and Spheres in Inner Product Space If V is an inner product space, then the set of points in V that satisfy ||u|| = 1 is called the unite sphere or sometimes the unit circle in V. In R2 and R3 these are the points that lie 1 unit away form the origin. 2008/12/17 Elementary Linear Algebra 13 Unit Circles in R2 Sketch the unit circle in an xy-coordinate system in R2 using the Euclidean inner product u, v= u1v1 + u2v2 Sketch the unit circle in an xy-coordinate system in R2 using the Euclidean inner product u, v= 1/9u1v1 + 1/4u2v2 Solution ½ 2 2 ½ If u = (x,y), then ||u|| = u, u = (x + y ) , so the equation of the unit circle is x2 + y2 = 1. ½ 2 2 ½ If u = (x,y), then ||u|| = u, u = (1/9x + 1/4y ) , so the equation of the unit circle is x2 /9 + y2/4 = 1. 2008/12/17 Elementary Linear Algebra 14 Inner Products Generated by Matrices u1 v 1 u2 v 2 Letu and v = be vectors in Rn (expressed as n1 :: un v n matrices), and let A be an invertible nn matrix. If u · v is the Euclidean inner product on Rn, then the formula u, v= Au · Av defines an inner product; it is called the inner product on Rn generated by A. Recalling that the Euclidean inner product u · v can be written as the matrix product vTu, the above formula can be written in the alternative form u, v= (Av) TAu, or equivalently, u, v= vTATAu 2008/12/17 Elementary Linear Algebra 15 Inner Product Generated by the Identity Matrix The inner product on Rn generated by the nn identity matrix is the Euclidean inner product: Let A = I, we have u, v= Iu · Iv = u · v The weighted Euclidean inner product u, v= 3u1v1 + 2u2v2 is the inner product on R2 generated by 3 0 A 0 2 since 3 0 3 0 u1 3 0u1 u,v v1 v2 v1 v2 3u1v1 2u2v2 0 20 2u2 0 2u2 2008/12/17 Elementary Linear Algebra 16 Inner Product Generated by the Identity Matrix In general, the weighted Euclidean inner product u, v= w1u1v1 + w2u2v2 + … + wnunvn is the inner product on Rn generated by 2008/12/17 Elementary Linear Algebra 17 An Inner Product on M22 u1 u 2 v 1 v 2 If UV and u3 u 4 v 3 v 4 are any two 22 matrices, then T T U, V= tr(U V) = tr(V U) = u1v1 + u2v2 + u3v3 + u4v4 defines an inner product on M22 For example, if 1 2 1 0 UV and 3 4 3 2 then U, V= 1(-1)+2(0) + 3(3) + 4(2) = 16 The norm of a matrix U relative to this inner product is 1/ 2 2 2 2 2 U U, U u1 u 2 u 3 u 4 and the unit sphere in this space consists of all 22 matrices U whose entries satisfy the equation ||U|| = 1, which on squaring yields 2 2 2 2 u1 + u2 + u3 + u4 = 1 2008/12/17 Elementary Linear Algebra 18 An Inner Product on P2 2 2 If p = a0 + a1x + a2x and q = b0 + b1x + b2x are any two vectors in P2, then the following formula defines an inner product on P2: p, q= a0b0 + a1b1 + a2b2 The norm of the polynomial p relative to this inner product is 1/ 2 2 2 2 p p, p a0 a 1 a 2 and the unit sphere in this space consists of all polynomials p in P2 whose coefficients satisfy the equation || p || = 1, which on squaring yields 2 2 2 a0 + a1 + a2 =1 2008/12/17 Elementary Linear Algebra 19 Theorem 6.1.1 (Properties of Inner Products) If u, v, and w are vectors in a real inner product space, and k is any scalar, then: 0, v= v, 0= 0 u, v + w= u, v+ u, w u, kv=k u, v u –v, w= u, w–v, w u, v –w= u, v–u, w 2008/12/17 Elementary Linear Algebra 20 Example u –2v, 3u + 4v = u, 3u + 4v–2v, 3u+4v = u, 3u+ u, 4v–2v, 3u–2v, 4v = 3 u, u+ 4 u, v–6 v, u–8 v, v = 3 || u ||2 + 4 u, v–6 u, v–8 || v ||2 = 3 || u ||2 –2 u, v–8 || v ||2 2008/12/17 Elementary Linear Algebra 21 Example We are guaranteed without any further proof that the five properties given in Theorem 6.1.1 are true for the inner product on Rn generated by any matrix A.