MAT641- MSC Mathematics, MNIT Jaipur
Functional Analysis-Inner Product Space and orthogonality
Dr. Ritu Agarwal Department of Mathematics, Malaviya National Institute of Technology Jaipur April 11, 2017
Inner product on linear space X over field K is defined as a function h.i : X × X −→ K which satisfies following axioms: For x, y, z ∈ X, α ∈ K, i. hx + y, zi = hx, zi + hy, zi; ii. hαx, yi = αhx, yi; iii. hx, yi = hy, xi; iv. hx, xi ≥ 0, hx, xi = 0 iff x = 0. An inner product defines norm as:
kxk = phx, xi and the induced metric on X is given by
d(x, y) = kx − yk = phx − y, x − yi
Further, following results can be derived from these axioms:
hαx + βy, zi = αhx, zi + βhy, zi
hx, αyi = αhx, yi hx, αy + βzi = αhx, yi + βhx, zi α, β are scalars and bar denotes complex conjugation. Remark 1.1. Inner product is linear w.r.to first factor and conjugate linear (semilin- ear) w.r.to second factor. Together, this property is called ‘Sesquilinear’ (means one and half-linear).
Date: April 11, 2017 Email:[email protected] Web: drrituagarwal.wordpress.com MAT641-Inner Product Space and orthogonality 2
Definition 1.2 (Hilbert space). A complete inner product space is known as Hilbert Space. Example 1. Examples of Inner product spaces
n n i. Euclidean Space R , hx, yi = ξ1η1 + ... + ξnηn, x = (ξ1, ..., ξn), y = (η1, ..., ηn) ∈ R . n n ii. Unitary Space C , hx, yi = ξ1η1 + ... + ξnηn, x = (ξ1, ..., ξn), y = (η1, ..., ηn) ∈ C . 2 R b iii. Function space L [a, b], hx, yi = a x(t)y(t)dt, x(t) and y(t) are complex valued functions (May be real valued also).
2 P∞ 2 P∞ iv. l = {x = (ξ1, ξ2, ...): i=1 |ξi| < ∞} is an IPS for hx, yi = i=1 ξiη¯i.
2 Norm and Parallelogram equality
To prove triangle inequality for normed space, first we are required to prove the Schwarz inequality. Theorem 2.1 (Schwarz inequality).
|hx, yi| ≤ kxkkyk (2.1) where, equality holds if and only if {x, y} is linearly dependent set. Proof. If y = 0, then (2.1) holds since hx, 0i = 0. Let y 6= 0. For every scalar α, we have 0 ≤ kx − αyk2 = hx − αy, x − αyi (2.2) = hx, xi − α¯hx, yi − α[hy, xi − α¯hy, yi]
hy,xi Choosing α = hy,yi so that the expression in brackets vanish. We get hy, xi |hx, yi|2 0 ≤ hx, xi − hx, yi = kxk2 − (Usinghy, xi = hx, yi) (2.3) hy, yi kyk2 Rearranging the terms, we get (2.1). Equality holds if kx − αyk = 0 i.e. x = αy, {x, y} are linearly dependent.
Theorem 2.2. All inner product spaces are normed spaces for the norm defined by kxk = phx, xi. Also, show that the triangle inequality for norm can be obtained from inner product as kx + yk ≤ kxk + kyk (2.4) where, equality holds if and only if {x, y} is linearly dependent set x = cy, c ≥ 0 or y = 0. MAT641-Inner Product Space and orthogonality 3
Proof. (i) From axiom (iv) for inner product space, we have kxk = phx, xi ≥ 0 and kxk = phx, xi = 0 iff x = 0. (ii) kαxk = phαx, αxi = pαα¯hx, xi = p|α|2hx, xi = |α| kxk. (iii) Triangle inequality
kx + yk2 = hx + y, x + yi = hx, xi + hx, yi + hy, xi + hy, yi (2.5) = kxk2 + hx, yi + hy, xi + kyk2
Using Schwarz inequality (2.1)
kx + yk2 ≤ kxk2 + 2|hx, yi| + kyk2 ≤ kxk2 + 2kxk kyk + kyk2 = (kxk + kyk)2 (2.6)
Taking square root on both sides, we get(2.4).
Further, equality holds in this derivation if and only if
hx, yi + hy, xi = 2kxk kyk i.e. hx, yi + hy, xi = 2
2.1 Parallelogram equality A norm on an inner product space satisfies the important Parallelogram equality
kx + yk2 + kx − yk2 = 2(kxk2 + kyk2) (2.7)
Proof. LHS = kx + yk2 + kx − yk2 = hx + y, x + yi + hx − y, x − yi = hx, xi + hx, yi + hy, xi + hy, yi + hx, xi − hx, yi + hy, yi − hy, xi (2.8) = 2(hx, xi + hy, yi) = 2(kxk2 + kyk2) MAT641-Inner Product Space and orthogonality 4
Examples of normed spaces which are not inner product spaces: Example 2. Sequence space lp, p 6= 2 Consider x = (1, 1, 0, 0, 0, ....) and y = (1, −1, 0, 0, 0, ...) ∈ lp and calculate
kxk = kyk = 21/p, kx + yk = kx − yk = 2 Parallelogram equality is not satisfied. Hence lp, p 6= 2 is a Banach space but not Hilbert space.
Example 3. Space C[a,b] is not an inner product space for kxk = maxt∈J |x(t)|, J = [a, b]. Consider x(t) = 1 and y(t) = (t − a)/(b − a). We have, kxk = 1, kyk = 1 and t−a t−a kx(t) + y(t)k = k1 + b−ak = 2, kx(t) − y(t)k = k1 − b−ak = 1. Thus, Parallelogram equality is not satisfied. Theorem 2.4 (Polarization identity). The inner product in term of induced norm is given by 1. For a real IPS X, 1 hx, yi = (kx + yk2 − kx − yk2) (2.9) 4 2. For a complex IPS 1 3 Continuity in IPS Theorem 3.1 (Continuity). If in an inner product space xn → x and yn → y, then hxn, yni → hx, yi. Proof. Consider |hxn, yni − hx, yi| = |hxn, yni − hxn, yi + hxn, yi − hx, yi| = |hxn, yn − yi| + |hxn − x, yi| Applying Schwarz inequality |hxn, yni − hx, yi| ≤ kxnk kyn − yk + kxn − xk kyk → 0 as n → ∞ Hence inner product is a continuous function. MAT641-Inner Product Space and orthogonality 5 Definition 3.2 (Isomorphism). An isomorphism T of an inner product space (X, h.i) ˜ onto an IPS (X, h.i0) over the same field K of scalars is a bijective linear operator ˜ T : X → X which preserves the values of the inner products i.e. hT x, T yi0 = hx, yi. Exercise 3.3. Let Y be a subspace of a Hilbert space H. 1. Y is complete iff Y is closed in H. 2. If Y is a finite dimensional subspace then Y is complete. 3. Every subspace of a separable space is separable. Theorem 3.4 (Minimizing Vector). Let X be an inner product space and Y 6= φ is a convex subset which is complete (in the metric induced by the inner product) subset. Then for every given x ∈ X, there is an unique y ∈ Y such that δ = inf kx − y0k = kx − yk. y0∈Y 0 Proof: (Existence of y): Given δ = infy0∈Y kx − y k. By definition of infimum, there exists a sequence (yn) in Y such that δn = kx − ynk δn → δ as n → ∞ We will show that (yn) is a Cauchy sequence in Y. Let x − yn = vn. Then (yn + ym) kvn + vmk = k2x − (yn + ym)k = 2 x − 2 But yn, ym ∈ Y ,(yn + ym)/2 ∈ Y as Y is convex. Hence kvn + vmk ≥ 2δ. (3.1) 2 2 2 2 2 Now kyn − ymk = kvn − vmk = 2kvnk + 2kvmk − kvn − vmk (by Parallelogram law). 2 2 2 2 2 2 2 ⇒ kyn − ymk = 2kx − ynk + 2kx − ymk − kvn − vmk ≤ 2δn + 2δm − 4δ (using (3.1)). As n, m → ∞, δn → δ and δm → δ. Thus kyn − ymk → 0 as n, m → ∞ i.e. (yn) is a Cauchy sequence. Since Y is complete, (yn) → y ∈ Y . Thus existence of y is guaranteed. (Uniqueness): Let y0 and y1 ∈ Y such that δ = kx − y0k = kx − y1k 2 2 2 . Now ky1 − y0k = k(y1 − x) − (y0 − x)k ≤ 2ky1 − xk + 2ky0 − xk − ky1 − x + y0 − xk (by Parallelogram law) 2 2 2 ⇒ ky1 − y0k ≤ 2δ + 2δ − 4δ → 0. But ky1 − y0k ≥ 0, we have ky1 − y0k = 0 i.e. y1 = y0. MAT641-Inner Product Space and orthogonality 6 4 Orthogonality Definition 4.1 (Orthogonality). An element x of an inner product space X is said to be orthogonal to an element y ∈ X if hx, yi = 0 and we write x⊥y. Theorem 4.2 (Orthogonality). Let X be an inner product space and Y 6= φ is a convex complete (in the metric induced by the inner product) subset. Then for a fixed x ∈ X, z = x − y is orthogonal to Y where kx − yk = δ = inf kx − y0k, y ∈ Y. y0∈Y Proof: Let z ⊥ Y is false. Then there exist a y0 ∈ Y such that hz, y0i = β 6= 0 Clearly, y0 6= 0 since otherwise hz, y0i = 0. Consider the norm kz − αy0k2 = hz − αy0, z − αy0i = hz, zi − αhz, y0i − α(hz, y0i − αhy0, y0i) hz,y0i β Choose α = hy0,y0i = ky0k2 so that ββ kz − αy0k2 = kzk2 − < kzk2 = δ2 ky0k2 We claim, that kz − αy0k2hδ2 can never happen. Since 0 0 kz − αy k = kx − y − αy k = kx − αy1k ≥ δ 0 y1 = y+αy and δ = infy∈Y kx−yk. This shows contradiction. Hence β = 0 ⇒ hz, yi = 0 i.e. (x − y) ⊥ Y . 4.1 Orthogonal Compliments and Direct sums Definition 4.3 (Direct Sum). A vector space X is said to be direct sum of two sub- spaces Y and Z of X, written X = Y ⊕ Z, if each x ∈ X has a unique representation x = y + z. Z is called algebraic complement of Y in X and vice versa and Y, Z is called a comple- mentary pair of subspaces in X. Definition 4.4 (Anihilator). An orthogonal compliment of Y 6= φ in an IPS X is the set Y ⊥ = {x ∈ X : x ⊥ Y }, called anihilator. MAT641-Inner Product Space and orthogonality 7 Theorem 4.5. (i) Y ⊥ is a vector subspace. (ii) Y ⊥ is a closed subspace. (iii) Y ⊂ Y ⊥⊥. Main concern is for a Hilbert space H, a closed subspace Y and its ’Orthogonal Compliment’ Y ⊥ = {z ∈ H|z ⊥ Y } Theorem 4.6 (Projection theorem). Let Y be any closed convex subspace of a Hilbert space H. Then H = Y ⊕ Y ⊥. Proof: Since H is complete and Y is closed, Y is complete. By Theorem 4.2, for every x ∈ H, there is a y ∈ Y such that x = y + z z ∈ Z = Y ⊥ To prove uniqueness, assume that x = y + z = y0 + z0, where y, y0 ∈ Y and z, z0 ∈ Z. Since y − y0 ∈ Y and z − z0 ∈ Z, we see that y − y0 ∈ Y ∩ Y ⊥ = {0}. This implies y = y0 and z = z0. Hence proved. Definition 4.7 (Orthogonal Projection). Define a mapping P : H → Y i.e. P x = y.P is called ‘Orthogonal Projection’ of H onto Y. Obviously P is a bounded linear operator. It maps H onto Y, Y onto itself and Z = Y ⊥ onto {0}. P is idempotent, i.e. P 2 = P . Lemma 4.8 (Null space). The orthogonal compliment Y ⊥ of a closed subspace Y of a Hilbert space H is the null space N(P ) of the orthogonal projection P of H onto Y . Proof. Since Y is a closed subspace of H, H = Y ⊕ Y ⊥. Therefore any x ∈ H can be written as x = y +z where y ∈ Y and z ∈ Y ⊥. If P : H → Y is the projection mapping, P x = y, x ∈ H. In particular let x ∈ Y ⊥ then x = 0 + z = z since hx, yi = 0. Hence P z = 0. z is arbitrary element of Y ⊥. Hence the theorem. Lemma 4.9 (Closed subspace). If Y is a closed subspace of a Hilbert space H, then Y = Y ⊥⊥ Proof. To prove Y = Y ⊥⊥, we need to show that Y ⊂ Y ⊥⊥ and Y ⊃ Y ⊥⊥. Let x ∈ Y ⇒ x ⊥ Y ⊥ ⇒ x ∈ Y ⊥⊥. That is Y ⊂ Y ⊥⊥. To show Y ⊃ Y ⊥⊥, let x ∈ Y ⊥⊥. Since Y ⊥⊥ is a vector space, y ∈ Y ⊂ Y ⊥⊥, z = x − y ∈ Y ⊥⊥. Hence z ⊥ Y ⊥ but z ∈ Y ⊥. Hence z = 0 so that x = y i.e. x ∈ Y . Since x is arbitrary, Y ⊃ Y ⊥⊥. Lemma 4.10 (Dense Set (3.3-7)). For any subset M 6= φ of a Hilbert space H, the span of M is dense in H if and only if M ⊥ = {0}. MAT641-Inner Product Space and orthogonality 8 Proof. Let x ∈ M ⊥ and assume V = span M to be dense in H. Then x ∈ V¯ = H. Thus, ⊥ ⊥ there exists a sequence (xn) in V such that xn → x. Since x ∈ M and M ⊥ V , we have hxn, xi = 0. The continuity of the inner product implies that hxn, xi → hx, xi = kxk2 = 0, so that x = 0. Since x ∈ M ⊥ is arbitrary, this shows that M ⊥ = {0}. Conversely, suppose that M ⊥ = {0}. If x ⊥ V =Span M, then x ⊥ M so that x ∈ M ⊥ and x = 0. Hence V ⊥ = {0}. H being Hilbert space, can be written as H = Y ⊕ Y ⊥, Y is closed. Noting that V is subspace of H, taking Y = V¯ , we thus obtain H = V¯ . 5 Orthonormal sets and sequences An orthogonal set M in an inner product space X is a subset M ⊂ X whose elements are pairwise orthogonal. An orthonormal set M in an inner product space X is a subset M ⊂ X is an orthog- onal set in X whose elements have norm 1. That is for all x, y ∈ M ( 0 x 6= y hx, yi = (5.1) 1 x = y If an orthogonal or orthonormal set M is countable, we can arrange it in a sequence (xn) and call it orthogonal or orthonormal sequence. For orthogonal elements x, y, we have hx, yi = 0 so that parallelogram equality leads to kx + yk2 = kxk2 + kyk2 (5.2) which is Pythogorean relation. More generally, if {x1, x2, ...xn} is an orthogonal set then 2 2 2 kx1 + ... + xnk = kx1k + ... + kxnk (5.3) Lemma 5.1 (Linear independence). An orthonormal set is linearly independent. Proof. Let {e1, e2, ..., en} be orthonormal. Consider the equation α1e1 + α2e2 + ... + αnen = 0 Taking inner product with ej, 1 ≤ j ≤ n, we get n n X X h αkek, eji = αkhek, eji = αjhej, eji = αj = 0 k=1 k=1 This proves the linear independence for any finite orthonormal set. Example 4. In the space R3 the basis (1,0,0), (0,1,0), (0,0,1) is an orthonormal basis. MAT641-Inner Product Space and orthogonality 9 2 Example 5. In Example 1(iv), e1, .., en, ... is the orthonormal sequence in l where e = (0, 0, ..., 1 , 0, ...) n |{z} nth position Example 6. Let X be the IPS of all real valued continuous functions on [0, 2π] with inner product defined by Z 2π hx, yi = x(t) y(t)dt (5.4) 0 An orthogonal sequences in X are un = cos nt and vn = sin√nt. That is hun, umi = 0 for cos√ nt sin√nt n 6= m. Also hvn, vmi = 0 for n 6= m. If we write e0 = u0/ 2π and en = π ,e ¯n π , we get an orthonormal sequence of functions {e0, en, e¯n}. 5.1 Advantage of orthonormal sequence Expressing elements of IP space X in terms of orthonormal basis actually makes the determination of coefficients very easy as compared to arbitrary linearly independent set. Let e1, .., en, ... is the orthonormal sequence in X. Then x ∈ X can be written as n X x = αkek k=1 n X hx, eji = αkhek, eji = αj k=1 With these coefficients, we can write n X x = hx, ekiek k=1 Theorem 5.2 (Bessel inequality). Let (ek) be an orthonormal sequence in an inner product space X. Then for every x ∈ X ∞ X 2 2 |hx, eki| ≤ kxk (5.5) k=1 The inner product hx, eki in (5.5) are called Fourier Coefficients of x with respect to orthonormal sequence (ek). Pn Proof. Let y = k=1 |αkek where n is fixed. Here αk = hy, eki as (ek) is orthonormal sequence. We claim that for a particular choice of αk = hx, eki, k = 1, 2, ..., n, we shall obtain a y ∈ Y such that z = x − y ⊥ Y . By orthonormality, n n n 2 X X X 2 kyk = h hx, ekiek, hx, ejieji = |hx, eki| k=1 j=1 k=1 MAT641-Inner Product Space and orthogonality 10 Using this, we can show that z ⊥ y ∈ Y as * n + n n X 2 X X 2 hz, yi = hx−y, yi = hx, yi−hy, yi = x, hx, ekiek −kyk = hx, ekihx, eki− |hx, eki| = 0 k=1 k=1 k=1 Using the Pythagorean relation kxk2 = kyk2 + kzk2 here, we get n 2 2 2 2 X 2 kzk = kxk − kyk = kxk − |hx, eki| k=1 Since kzk2 ≥ 0, for n ∈ N, we have n 2 X 2 kxk ≥ |hx, eki| k=1 Gram-Schmidt orthogonalization process 6 Total orthonormal sets Definition 6.1. A total set (or fundamental set) in a normed space X is a subset M ⊂ X whose span is dense in X. Accordingly an orthonormal set (or sequence or family) in an IPS X which is total in X is called a total orthonormal set (or sequence or family) in X. Remark 6.2. M is total in X if and only if span M = X. Remark 6.3. A total orthonormal family in X is sometimes called an orthonormal basis for X but not in sense of algebra except for the finite dimensional spaces. Lemma 6.4. In every Hilbert space H 6= {0}, there exists a total orthonormal set. Proof: For a finite dimensional space, this is clear. For an infinite dimensional separable space H, it follows from Gram-Schmidt orthogonalization process. For a non-separable H, a non-constructive proof follows from Zorn’s lemma. Definition 6.5 (Hilbert Dimension). All total orthonormal sets in a given Hilbert space H 6= {0} have the same cardinality. The latter is called the ‘Hilbert dimension’ or orthogonal dimension of H. Theorem 6.6 (Totality). Let M be a subset of IPS X. Then MAT641-Inner Product Space and orthogonality 11 (a) If M is total in X, then there does not exist a non-zero x ∈ X which is orthogonal to every element of M. Briefly x ⊥ M ⇒ x = 0 (6.1) (b) If X is complete, (6.1) is sufficient for the totality of M in X. Proof. (a) Let H be completion of X [1, Theorem 3.2-3]. Then X is regarded as a dense subspace of H. By assumption, M is total in X so that span M is dense in X and hence dense in H. Lemma 4.10, orthogonal compliment of M in H is {0}. Therefore, if x ∈ X and x ⊥ M then x = 0. (b) If X is a Hilbert space and M satisfies the given condition M ⊥ = {0}, by Lemma 4.10, the span of M is dense in X i.e. M is total in X. Remark 6.7. The completeness of X in (b) is essential. If X is not complete, there may not exist an orthonormal set M ⊂ X such that M is total in X. An example was given by J. Diximier (1953). Theorem 6.8 (Parseval relation). An orthonormal set M in a Hilbert space H is total in H if and only if for all x ∈ H, the following Parseval relation holds 2 X 2 kxk = |hx, eki| (6.2) k=1 Proof. If M is not total, there is a non-zero x ⊥ M in H. Since x ⊥ M, in (6.2), we have 2 hx, eki = 0 for all k, so that the RHS in (6.2) is zero whereas kxk 6= 0. This shows that (6.2) does not hold. Conversely assume that M is total in H. Consider any x ∈ H and its non-zero Fourier coefficients arranged in a sequence hx, e1i, hx, e2i, .... We now define y as X y = hx, ekiek, (6.3) k k may be finite or infinite as convergence follows from [1, Theorem 3.5-2]. Let us show that x − y ⊥ M. Using the orthonormality, for every ej in (6.3), we have X hx − y, eji = hx, eji − hx, ekihek, eji = hx, eji − hx, eji = 0. k For every v ∈ M not in (6.3), we have hx, vi = 0 and hek, vi = 0 so that X hx − y, vi = hx, vi − hx, ekihek, vi = 0 − 0 = 0. k Hence x − y ⊥ M i.e. x − y ∈ M ⊥. Since M is total in H, M ⊥ = {0}. This implies x − y = 0 or x = y. Hence * + 2 X X X kxk = hx, ekiek, hx, ekiek = hx, ekihx, eki (6.4) k k k This completes the proof. MAT641-Inner Product Space and orthogonality 12 Theorem 6.9. Let H be a Hilbert space. Then (a) If H is separable, every orthonormal set is countable. (b) If H contains an orthonormal sequence which is total in H, then H is separable. Proof. (a) Let H be separable References [1] Erwin Kreyszig, Introductory Functional Analysis With Applications, Wiley India Edition, 1989. [2] Walter Rudin, Functional Analysis, Tata Mc-Graw Hill, New Delhi. [3] Balmohan Vishnu Limaye, Functional analysis, New Age International, 1996. [4] Aldric Loughman Brown and A. Page, Elements of functional analysis, Van Nostrand- Reinhold, 1970. [5] J.B. Conway, A Course in Functional Analysis, 2nd Edition, Springer, Berlin, 1990. Contents 1 Inner Product Space 1 2 Norm and Parallelogram equality 2 2.1 Parallelogram equality ...... 3 3 Continuity in IPS 4 4 Orthogonality 6 4.1 Orthogonal Compliments and Direct sums ...... 6 5 Orthonormal sets and sequences 8 5.1 Advantage of orthonormal sequence ...... 9 6 Total orthonormal sets 10