Inner Products Definition An inner product on a vector space V over F is a function that assigns a scalar x, y for every x, y V, such that for all Chapter 6 – Inner Product Spaces h i ∈ x, y, z V and c F : ∈ ∈ (a) x + z, y = x, y + z, y Per-Olof Persson h i h i h i (b) cx, y = c x, y [email protected] h i h i (c) x, y = y, x (complex conjugation) h i h i Department of Mathematics (d) x, x > 0 if x = 0 University of California, Berkeley h i 6 Math 110 Linear Algebra Example n For x = (a1, . , an) and y = (b1, . , bn) in F , the standard inner product is defined by n x, y = a b h i i i Xi=1 Inner Products Properties of Inner Product Spaces Example A vector space V over F with a specific inner product is called an For f, g V = C([0, 1]), an inner product is given by inner product space. If F = C, V is a complex inner product ∈ 1 f, g = f(t)g(t) dt. space, and if F = R, V is a real inner product space. h i 0 R Definition Theorem 6.1 For an inner product space V, x, y, z V, and c F : The conjugate transpose or adjoint of A Mm n(F ) is the n m ∈ ∈ ∈ × × (a) x, y + z = x, y + x, z matrix A∗ such that (A∗)ij = Aji for all i, j. h i h i h i (b) x, cy = c x, y h i h i Example (c) x, 0 = 0, x = 0 h i h i The Frobenius inner product on V = Mn n(F ) is defined by (d) x, x = 0 if and only if x = 0 × h i A, B = tr(B∗A) for A, B V. (e) If x, y = x, z for all x V, then y = z h i ∈ h i h i ∈ Norms Orthogonality Definition Definition Let V be an inner product space. For x V, the norm or the ∈ Let V be an inner product space. Vectors x, y V are orthogonal length of x is x = x, x . ∈ k k h i (perpendicular) if x, y = 0. A subset S of V is orthogonal if any h i p two distinct vectors in S are orthogonal. A vector x V is a unit Example ∈ vector if x = 1. A subset S of V is orthonormal if S is For x = (a , . , a ) V = Fn, the Euclidean length is the norm k k 1 n ∈ orthogonal and consists entirely of unit vectors. n 1/2 x = a 2 k k | i| Example "i=1 # X Consider the inner product space H of continuous complex-valued Theorem 6.2 functions defined on [0, 2π] with the inner product For an inner product space V over F and all x, y V, c F : 1 2π ∈ ∈ f, g = f(t)g(t) dt. (a) cx = c x h i 2π k k | | · k k Z0 (b) x = 0 if and only if x = 0. In any case, x 0. k k k k ≥ Let f (t) = eint for any integer n, where 0 t 2π. Then (c) (Cauchy-Schwarz Inequality) x, y x y n ≤ ≤ |h i| ≤ k k · k k S = fn : n is an integer is an orthonormal subset of H. (d) (Triangle Inequality) x + y x + y { } k k ≤ k k k k Orthonormal Bases Gram-Schmidt Orthogonalization Definition A subset of an inner product space V is an orthonormal basis for V Theorem 6.4 if it is an ordered basis that is orthonormal. Let V be an inner product space and S = w , . , w a linearly { 1 n} independent subset of V. Define S = v , . , v , where v = w Theorem 6.3 0 { 1 n} 1 1 and Let V be an inner product space and S = v , . , v an { 1 k} orthogonal subset of V consisting of nonzero vectors. If k 1 − k wk, vj y, vi vk = wk h 2 ivj for 2 k n. − vj ≤ ≤ y span(S), then y = h 2ivi. j=1 ∈ vi X k k Xi=1 k k Then S0 is an orthogonal set of nonzero vectors such that Corollary 1 span(S0) = span(S). If in addition S is orthonormal, then y = k y, v v . i=1h ii i This construction of v , . , v is called the Gram-Schmidt P { 1 n} Corollary 2 process. Let V be an inner product space, and S an orthogonal subset of V consisting of nonzero vectors. Then S is linearly independent. Representations in Orthonormal Bases Orthogonal Complement Theorem 6.5 Definition Let V be a nonzero finite-dimensional inner product space. Then V Let S be a nonempty subset of an inner product space V. We has an orthonormal basis β. Furthermore, if β = v1, . , vn and define S to be the set of all vectors in V that are orthogonal to { } ⊥ x V, then every vector in S; that is, S⊥ = x V : x, y = 0 for all y S . ∈ n { ∈ h i ∈ } The set S is called the orthogonal complement of S. x = x, v v . ⊥ h ii i i=1 X Theorem 6.6 Corollary Let W be a finite-dimensional subspace of an inner product space V, and let y V. Then there exist unique vectors u W and Let V be a finite-dimensional inner product space with an ∈ ∈ orthonormal basis β = v , . , v . Let T be a linear operator on z W⊥ such that y = u + z. Furthermore, if v1, . , vk is an { 1 n} ∈ k { } V, and let A = [T] . Then for any i and j, A = T(v ), v . orthonormal basis for W, then u = i=1 y, vi vi. β ij h j ii h i P Definition Corollary Let β be an orthonormal subset (possibly infinite) of an inner The vector u in Thm 6.6 is the unique vector in W that is “closest” product space V, and let x V. The Fourier coefficients of x to y; that is, for any x W, y x y u , and this inequality ∈ ∈ k − k ≥ k − k relative to β are the scalars x, y , where y β. is an equality if and only if x = u (the orthogonal projection). h i ∈ Orthogonal Extension The Adjoint Theorem 6.8 Theorem 6.7 Let V be a finite-dimensional inner product space over F , and let Suppose that S = v , . , v is an orthonormal set in an g : V F be a linear transformation. Then there exists a unique { 1 k} → n-dimensional inner product space V. Then vector y V such that g(x) = x, y for all x V. ∈ h i ∈ (a) S can be extended to an orthonormal basis Theorem 6.9 v1, . , vk, vk+1, . , vn for V. { } Let V be a finite-dimensional inner product space, and let T be a (b) If W = span(S), then S1 = vk+1, . , vn is an orthonormal { } linear operator on V. Then there exists a unique function basis for W⊥. T : V V such that T(x), y = x, T (y) for all x, y V. (c) If W is any subspace of V, then ∗ → h i h ∗ i ∈ Furthermore, T∗ is linear. dim(V) = dim(W) + dim(W⊥). The linear operator T∗ is called the adjoint of T. Properties of Adjoints Properties of Adjoints Theorem 6.10 Let V be a finite-dimensional inner product space, and let β be an orthonormal basis for V. If T is a linear operator on V, then [T∗]β = [T]β∗ . Corollary Let A, B be n n matrices. Then Corollary × (a) (A + B)∗ = A∗ + B∗ Let A be an n n matrix. Then LA = (LA)∗. × ∗ (b) (cA)∗ = cA∗ for all c F ∈ Theorem 6.11 (c) (AB)∗ = B∗A∗ Let V be an inner product space, and T, U linear operators on V. (d) A∗∗ = A (e) I = I (a) (T + U)∗ = T∗ + U∗ ∗ (b) (cT) = cT for any c F ∗ ∗ ∈ (c) (TU)∗ = U∗T∗ (d)T ∗∗ = T (e)I ∗ = I Normal Operators Properties of Normal Operators Lemma Theorem 6.15 Let T be a linear operator on a finite-dimensional inner product With V an inner product space and T a normal operator on V: space V. If T has en eigenvector, then so does T . (a) T(x) = T (x) for all x V. ∗ k k k ∗ k ∈ (b)T cI is normal for every c F . − ∈ Theorem 6.14 (Schur) (c) If x is an eigenvector of T, then x is also an eigenvector of With T as in the lemma, suppose that the characteristic T∗. In fact, if T(x) = λx, then T∗(x) = λx. polynomial of T splits. Then there exists an orthonormal basis β (d) If λ1, λ2 are distinct eigenvalues of T then the corresponding for V such that the matrix [T] is upper triangular. β eigenvectors x1, x2 are orthogonal. Definition Theorem 6.16 Let V be an inner product space and T a linear operator on V. T is Let T be a linear operator on a finite-dimensional complex inner normal if TT = T T. An n n real or complex matrix A is ∗ ∗ × product space V. Then T is normal if and only if there exists an normal if AA∗ = A∗A. orthonormal basis for V consisting of eigenvectors of T. Self-Adjoint Operators Unitary and Orthogonal Operators Definition Definition Let T be a linear operator on an inner product space V.