6 Inner Product Spaces
Total Page:16
File Type:pdf, Size:1020Kb
Lectures 16,17,18 6 Inner Product Spaces 6.1 Basic Definition Parallelogram law, the ability to measure angle between two vectors and in particular, the concept of perpendicularity make the euclidean space quite a special type of a vector space. Essentially all these are consequences of the dot product. Thus, it makes sense to look for operations which share the basic properties of the dot product. In this section we shall briefly discuss this. Definition 6.1 Let V be a vector space. By an inner product on V we mean a binary operation, which associates a scalar say u, v for each pair of vectors u, v) in V, satisfying h i the following properties for all u, v, w in V and α, β any scalar. (Let “ ” denote the complex − conjugate of a complex number.) (1) u, v = v, u (Hermitian property or conjugate symmetry); h i h i (2) u, αv + βw = α u, v + β u, w (sesquilinearity); h i h i h i (3) v, v > 0 if v =0 (positivity). h i 6 A vector space with an inner product is called an inner product space. Remark 6.1 (i) Observe that we have not mentioned whether V is a real vector space or a complex vector space. The above definition includes both the cases. The only difference is that if K = R then the conjugation is just the identity. Thus for real vector spaces, (1) will becomes ‘symmetric property’ since for a real number c, we have c¯ = c. (ii) Combining (1) and (2) we obtain (2’) αu + βv, w =α ¯ u, w + β¯ v, w . In the special case when K = R, (2) and (2’) h i h i h i together are called ‘bilinearity’. Example 6.1 (i) The dot product in Rn is an inner product. This is often called the standard inner product. t t Cn n ∗ (ii) Let x =(x1, x2,...,xn) and y =(y1,y2,...,yn) . Define x, y = i=1 xiyi = x y. ∈Cn h i This is an inner product on the complex vector space . P (iii) Let V = C[0, 1]. For f,g V, put ∈ 1 f, g = f(t)g(t) dt. 0 h i Z It is easy to see that f, g is an inner product. h i 50 6.2 Norm Associated to an Inner Product Definition 6.2 Let V be an inner product space. For any v V, the norm of v, denoted by v , is the ∈ k k positive square root of v, v : v = v, v . h i k k h i q For standard inner product in Rn, v is the usual length of the vector v. k k Proposition 6.1 Let V be an inner product space. Let u, v V and c be a scalar. Then ∈ (i) cu = c u ; k k | |k k (ii) u > 0 for u = 0; k k 6 (iii) u, v u v ; |h i|≤k kk k (iv) u + v u + v , equality holds in this triangle inequality iff u, v are linearly k k≤k k k k dependent over R. Proof: (i) and (ii) follow from definitions 6.1,6.2. (iii) This is called the Cauchy-Schwarz inequality. If either u or v is zero, it is clear. So let u be nonzero. Consider the vector w = v au. − where a is a scalar to be specified later. Then 0 w, w = v au, v au ≤h i h − − i = v, v v, au au, v + au, au h i−h i−h i h i = v, v a v, u a u, v + aa u, u h i − h i − h i h i Substitute a = u, v / u, u and multiply by u, u , we get u, v 2 u, u v, v . h i h i h i |h i| ≤ h ih i This proves (iii). (iv) Using (iii), we get ( u, v ) u v . Therefore, ℜ h i ≤k kk k u + v 2 = u 2 + u, v + v, u + v 2 k k k k h i h i k k = u 2 +2 u, v + v 2 k k ℜh i k k u 2 +2 u v + v 2 ≤ k k k kk k k k = ( u + v )2. k k k k Thus u + v u + v . k k≤k k k k Now equality holds in (iv) iff u, v = u, v and equality holds in (iii). This is the ℜh i h i same as saying that u, v is real and w = 0. This is the same as saying that v = au and h i v, u = a u, u is real. Which is the same as saying that v is a real multiple of u. h i h i ♠ 6.3 Distance and Angle Definition 6.3 In any inner product space V, the distance between u and v is defined as d(u, v)= u v . k − k 51 Definition 6.4 Let V be a real inner product space. The angle between vectors u, v is defined to be the angle θ, 0 θ π so that ≤ ≤ u, v cos θ = h i . u v k kk k Proposition 6.2 (i) d(u, v) 0 and d(u, v)=0 iff u = v; ≥ (ii) d(u, v)= d(v, u); (iii) d(u, w) d(u, v)+ d(v, w). ≤ 6.4 Gram-Schmidt Orthonormalization n The standard basis E = e1, e2,...,en of R has two properties : { } (i) each ei has length one (ii) any two elements of E are mutually perpendicular. When vectors are expressed in terms of the standard basis, calculations become easy due to the above properties. Now, let V be any inner product space of dimension d. We wish to construct an analogue of E for V. Definition 6.5 Two vectors u, v of an inner product space V are called orthogonal to each other or perpendicular to each other if u, v =0. We express this symbolically by u v. A h i ⊥ set S consisting of non zero elements of V is called orthogonal if u, v =0 for every pair of h i elements of S. Further if every element of V is of unit norm then S is called an orthonormal set. Further, if S is a basis for V then it is called an orthonormal basis. One of the biggest advantage of an orthogonal set is that Pythagorus theorem and its general form are valid, which makes many computations easy. Theorem 6.1 Pythagorus Theorem: Given w, u V such that w u we have ∈ ⊥ w + u 2 = w 2 + u 2. (47) k k k k k k Proof: We have, w + u 2 = w + u, w + u = w, w + u, w + w, u + u, u = w 2 + u 2 k k h i h i h i h i h i k k k k since u, w = w, u =0. h i h i ♠ Proposition 6.3 Let S = u1, u2,..., un be an orthogonal set. Then S is linearly inde- { } pendent. Proof: Suppose c1,c2,...,cn are scalars with c1u1 + c2u2 + . + cnun =0. Take inner product with ui on both sides to get ci ui, ui =0. h i Since ui = 0, we get ci =0. Thus U is linearly independent. 6 ♠ 52 Definition 6.6 If u and v are vectors in an inner product space V and v = 0 then the 6 vector v v, u h i v 2 k k is called the orthogonal projection of u along v. Remark 6.2 The order in which you take u and v matters here when you are working over complex numbers because of the inner product is linear in the second slot and conjugate linear in the first slot. Over the real numbers, however, this is not a problem. Theorem 6.2 (Gram-Schmidt Orthonormalization Process): Let V be any inner product space. Given a linearly independent subset u1,..., un there exists an orthonormal { } set of vectors v1,..., vn such that L( u1,..., uk ) = L( v1,..., vk ) for all 1 k n. { } { } { } ≤ ≤ In particular, if V is finite dimensional, then it has an orthonormal basis. Proof: The construction of the orthonormal set is algorithmic and of course, inductive. First we construct an intermediate orthogonal set w1,..., wn with the same property for each { } k. Then we simply ‘normalize’ each element, viz, take wi vi = . wi k k That will give us an orthonormal set as required. The last part of the theorem follows if we begin with u1,..., um as any basis for V. { } Take w1 := u1. So, the construction is over for k = 1. To construct w2, subtract from u2, its orthogonal projection along w1 Thus w1 w2 = u2 w1, u2 2 . −h i w1 k k u2 w1 w2 Check that w2, w1 =0. Thus w1, w2 is an orthogonal set. Check also that L( u1, u2 )= h i { } { } L( w1, w2 ). Now suppose we have constructed wi for i k < n as required. Put { } ≤ k wj wk+1 = uk+1 wj, uk+1 2 . − j=1h i wj X k k Now check that wk+1 is orthogonal to wj, j k and L( w1, w2,..., wk+1 )= L( u1,..., uk ). ≤ { } { } By induction, this completes the proof. ♠ 53 Example 6.2 Let V = P3[ 1, 1] denote the real vector space of polynomials of degree − − at most 3 defined on [ 1, 1] together with the zero polynomial.