6Inner-Pt1.Pdf
Total Page:16
File Type:pdf, Size:1020Kb
6 Inner Product Spaces You should be familiar with the scalar or dot product in R2, and with its basic properties. For any x1 y1 vectors x = ( x2 ) and y = y2 , define x · y := x1y1 + x2y2 p The norm or length of a vector is jjxjj = x · x, The angle q between vectors satisfies x · y = jjxjj jjyjj cos q, Vectors are orthogonal or perpendicular precisely when x · y = 0. The dot product is precisely what allows us to compute lengths of and angles between vectors in R2. An inner product is an algebraic structure that generalizes this idea to other vector spaces. 6.1 Inner Products and Norms Definition 6.1. An inner product on a vector space V over R is a positive definite, symmetric, bilinear form; that is, a function h , i : V × V ! R satisfying the following properties: 8l 2 R, x, y, z 2 V, (a) Positive definiteness x 6= 0 =) hx, xi > 0 (b) Symmetry hy, xi = hx, yi (c) Bilinearity hlx + y, zi = l hx, zi + hy, zi and hz, lx + yi = l hz, xi + hz, yi A real vector space equipped with an inner product is a (real) inner product space. p The norm or length of x 2 V is jjxjj := hx, xi.A unit vector has jjxjj = 1. Vectors x, y are perpendicular or orthogonal if hx, yi = 0. Vectors are orthonormal if they are orthogonal unit vectors. The bilinearity condition is worth unpacking slightly: • An inner product is linear in both slots: if z 2 V is constant, then we have two linear maps Tz,Uz : V ! R defined by Tz(x) = hx, zi ,Uz(x) = hz, xi • We need only assume the first linearity condition: that the inner product is linear in the second slot (Uz linear) follows from the symmetry condition. Euclidean Space The standard or Euclidean inner product on Rn is the usual dot product, v n u n h i = T = = + ··· + jj jj = u 2 x, y y x ∑ xjyj x1y1 xnyn, x t∑ xj j=1 j=1 where the norm is the Euclidean/Pythagorean length function. The resulting inner product space is often called Euclidean space. Unless indicated, if we refer to Rn as an inner product space, then we always mean with respect to the standard inner product. There are, however, many other ways to make Rn into an inner product space. 1 Example 6.2. (Weighted inner products) Define an alternative inner product on R2 via hx, yi = x1y1 + 3x2y2 It is easy to check that this satisfies the required properties: 2 2 (a) Positive definiteness If x 6= 0, then hx, xi = x1 + 3x2 > 0; (b) Symmetry follows from the commutativity of multiplication in R: hy, xi = y1x1 + 3y2x2 = x1y1 + 3x2y2 = hx, yi (c) Bilinearity follows from symmetry and the associative/distributive laws in R: hlx + y, zi = (lx1 + y1)z1 + 3(lx2 + y2)z2 = l(x1z1 + 3x2z2) + (y1z1 + 3y2z2) = l hx, zi + hy, zi With respect to this new inner product, the concept of orthogonality in R2 has changed: for example, if x = 1 1 and y = p1 3 , we see that fx, yg is an orthonormal set with respect to h , i: 2 1 2 3 −1 1 1 1 jjxjj2 = (12 + 3 · 12) = 1, jjyjj2 = (32 + 3 · 12) = 1, hx, yi = p (3 − 3) = 0 4 12 4 3 With respect to the standard dot product, hwoever, these vectors are not at all special: 1 5 1 x · x = , y · y = , x · y = p 2 6 2 3 We have the same vector space R2, but different inner product spaces: (R2, ·) 6= (R2, h , i). This example is easily generalized: let a1,..., an be positive real numbers (called weights) and define n hx, yi = ∑ ajxjyj = a1x1y1 + ··· + anxnyn j=1 It is a simple exercise to check that this defines an inner product on Rn. In particular, this means that Rn may be viewed as an inner product space in infinitely many distinct ways. There are even more inner products on Rn which aren’t described in the above example. For instance, if A 2 Mn(R) is a symmetric matrix, then hx, yi := yT Ax certainly satisfies the symmetry and bilinearity conditions. The matrix is called positive definite if hx, yi is an inner product. Here is an example of an inner product: 3 1 A = hx, yi = 3x y + x y + x y + x y 1 1 1 1 1 2 2 1 2 2 2 Basic properties of real inner products Lemma 6.3. Let V be a real inner product space, let x, y, z 2 V and l 2 R. 1. hx, 0i = 0 2. jjxjj = 0 () x = 0 3. jjlxjj = jlj jjxjj 4. hx, yi = hx, zi for all x =) y = z 5. (Cauchy–Schwarz inequality) jhx, yij ≤ jjxjj jjyjj with equality () x and y are parallel 6. (Triangle Inequality) jjx + yjj ≤ jjxjj + jjyjj with equality () x and y are parallel and point in the same direction Be careful with notation: jlj means absolute value (applies to scalars), while jjxjj means norm (applies to vectors). Proof. We leave parts 1–3 as exercises. 4. Let x = y − z and apply the bilinearity condition and part 2 to see that hx, yi = hx, zi =) 0 = hy − z, y − zi = jjy − zjj2 =) y = z 5. If y = 0, the result is trivial. WLOG we may assume jjyjj = 1: if the inequality holds for this, then it holds for all non-zero y by parts 1 and 4. Now expand: 0 ≤ jjx − hx, yi yjj2 (positive definiteness) = jjxjj2 + jhx, yij2 jjyjj2 − hx, yi hx, yi − hx, yi hy, xi (bilinearity) = jjxjj2 − jhx, yij2 (symmetry) Taking square-roots establishes the inequality. By part 2, equality holds if and only if x = hx, yi y, which is precisely when x, y are linearly dependent. 6. We establish the squared result: 2 jjxjj + jjyjj − jjx + yjj2 = jjxjj2 + 2 jjxjj jjyjj + jjyjj2 − jjxjj2 − hx, yi − hy, xi − jjyjj2 = 2jjxjj jjyjj − hx, yi ≥ 2jjxjj jjyjj − jhx, yij ≥ 0 (Cauchy–Schwarz) Equality requires both equality in Cauchy–Schwarz (x, y parallel) and that hx, yi ≥ 0: since x, y are already parallel, this means that one is a non-negative multiple of the other. 3 Complex Inner Product Spaces We can also define an inner product on a complex vector space. Since we will be primarily interested in the real case, we will often simply state the differences, if there are any. Definition 6.4. An inner product on a vector space V over C is a positive definite, conjugate-symmetric, sesquilinear form; a function h , i : V × V ! C satisfying the following properties: 8l 2 C, x, y, z 2 V, (a) Positive definiteness x 6= 0 =) hx, xi > 0 (b) Conjugate-symmetry hy, xi = hx, yi (c) Sesquilinearity hlx + y, zi = l hx, zi + hy, zi and hz, lx + yi = l hz, xi + hz, yi A complex vector space equipped with an inner product is a (complex) inner product space. The concepts of norm and orthogonality are identical to the real case. • The prefix sesqui- means one-and-a-half, reflecting the property of being linear in the first slot and conjugate-linear in the second. • A separate definition for the real case is not required. Simply replace C with F above: if F = R, we recover Definition 6.1. Note that an inner product space can only have base field R or C. For the remainder of this chapter, when we say an inner product space over F, we implicitly assume that F is R or C. • We need only assume that a complex inner product is linear in its first slot; conjugate-symmetry then demonstrates the conjugate-linearity property. • The entirety of Lemma 6.3 holds in a complex inner product space. A little extra care is needed for the proofs of the Cauchy–Schwarz and triangle inequalities. Note that jlj now means the complex modulus of l 2 C. • Warning! In case you dabble in the dark arts, be aware that the convention in Physics is for an inner product to be conjugate-linear in the first entry rather than the second! The standard (Hermitian) inner product and norm on Cn are v n u n ∗ u 2 hx, yi = y x = ∑ xjyj = x1y1 + ··· + xnyn, jjxjj = t∑ xj j=1 j=1 where y∗ = yT denotes the conjugate transpose of y. We can also define weighted inner products exactly as in Example 6.2: n hx, yi = ∑ ajxjyj = a1x1y1 + ··· + anxnyn j=1 note that the weights aj must still be positive real numbers. 4 Further Examples of Inner Products The Frobenius inner product If F = R or C, and A, B 2 Mm×n(F), define hA, Bi = tr B∗ A ∗ T where tr is the trace of a square matrix and B = B is the conjugate transpose. This makes Mm×n(F) into an inner product space. Example 6.5. In M3×2(C), *0 1 i 1 0 0 7 1+ 0 1 i 1 0 1 3 + 2i 2 − i −3 − 2i @2 − i 0 A , @ 1 2iA = tr @2 − i 0 A = tr 7 −2i 4 9 − 4i −4 + 7i 0 −1 3 − 2i 4 0 −1 = 2 − i − 4 + 7i = −2 − 8i 0 1 2 0 1 1 i 1 i 1 2 + i 0 6 i @2 − i 0 A = tr @2 − i 0 A = tr = 8 −i 0 −1 −i 2 0 −1 0 −1 This isn’t really a new example: if we map A ! Fm×n by stacking the columns of A, then the Frobenius inner product becomes the standard inner product.