HILBERT SPACES 1. Orthogonality Let M Be a Subspace of A

Home , Linear subspace, Orthogonal complement

FRANZ LUEF

1. Orthogonality Let M be a subspace of a Hilbert space H. Then the orthogonal complement of M is deﬁned by M ⊥ = {x ∈ H : hx, yi = 0 for all y ∈ M}. The linearity and the continuity of the inner product allow us to show the following fact. Lemma 1.1. M ⊥ is a closed subspace.

⊥ ⊥ Proof. Let x be an element of the closure of M . Hence there is a sequence (xn) in M ⊥ such that xn → x. Now, for any y ∈ M we have hx, yi = hlimn xn, yi = limnhxn, yi = ⊥ 0. Hence x ∈ M . Some elementary properties are (1) M ⊂ N implies that N ⊥ ⊂ M ⊥. (2) M ⊆ M ⊥⊥. A theorem of utmost importance is the closest point theorem for closed convex sets in Hilbert spaces. Theorem 1.2. If M is a closed convex set and x is any point of a Hilbert space H, then there is a unique point x0 ∈ M which is closest to x: For all x ∈ H we have that

kx − x0k ≤ kx − yk for all y ∈ M. ∞ This is not true in Banach spaces. For example, c0 is a closed subspace in ` , but there is no closest sequence in c0 to the sequence (1, 1, 1, ...). In fact, the distance between c0 and (1, 1, 1, ...) is 1, and this is achieved by any bounded sequence (an) with an ∈ [0, 2].

Important application: Let M be a closed subspace of H. In this situation, we are able to express the least distance property in a geometrical manner.

Theorem 1.3 (Projection theorem). For a closed subspace M of H the point x0 ∈ M ⊥ is the closest point to an element x ∈ H if and only if x − x0 ∈ M . Corollary 1.4. H = M ⊕ M ⊥.

Date: 10.09.2015.

1 Corollary 1.5. If M is a closed linear subspace of H, then M ⊥⊥ = M. If S is a set in H, then S⊥⊥ = span(S).

⊥⊥ ⊥ Proof. Let x ∈ M . Then x = x0 + y0 where x0 ∈ M and y0 ∈ M . We have 2 0 = hx, y0i = hx0, y0i + hy0, y0i = ky0k . Hence y0 = 0 and consequently, x ∈ M. Corollary 1.6. If S is a set in H, then S⊥⊥ = span(S). In particular, span(S) is dense in H if and only if S⊥ = 0. Proof. Note that span(S) is the smallest closed subspace containing S. Therefore, ⊥ since S ⊆ span(S), we get by the preceding corollary that span(S) ⊆ S⊥ and hence ⊥ ⊥⊥ ⊥ S ⊆ span(S) = span(S).

We look at the mapping PM that takes a x ∈ H to its closest point x0 ∈ M. Theorem 1.7. If M is a closed subspace of H, then the map P : H → H deﬁned by ⊥ P x = x0 is a continuous projection with im(P ) = M, ker(P ) = M and ker(P ) ⊥ im(P ). Proof. By deﬁnition of P , for any x ∈ H we have x − P x ∈ M ⊥ and so P x ∈ M.

The mapping P is linear: We have x + y − (P x + P y) = x − P x + y − P y ∈ M ⊥ and P x + P y ∈ M. Hence P (x + y) = P x + P y. We also have P (αx) = αP x.

P is a projection: P 2 = P . Since P x ∈ M we have P 2x = P x.

P is onto M: Since for any x ∈ M we have P x = x.

Moreover, x ∈ ker(P ) if and only if P x = 0 if and only if x = x − P x ∈ M ⊥.

P is continuous: Since kxk2 = kx − P xk2 + kP xk2 by Pythagoras. Hence kP xk ≤ kxk. Look at the following example: Let M be the subspace of H spanned by an orthonormal vector y. Then the orthogonal projection is given by P x = hx, yiy.

Another example: Find the quadratic polynomial closest to a function f in L2[0, 1]. 2 The space of quadratic polynomials P2 is three-dimensional and {1, x, x } is a basis 2 and P2 is a closed subspace of L [0, 1]. Hence by our general result, there exists a p ∈ P2 closest to f. In fact, p is characterized by the condition p − f ⊥ q for all q ∈ M. Equivalently, hp, xii = hf, xii for i = 0, 1, 2. The right hand sides can be computed, because f is a given function. Hence we get three linear equations in three unknowns, which can be solved.

2. Orthonormal Bases

A set of elements {en : n ∈ N} in H is called orthonormal if hei, eji = δij. We call it a basis if span{ei : ı ∈ N} is dense in H.

2 2 (1) ` has a basis consisting of {ei : i = 1, 2, ..} for ei = (0, 0, 0, ..., 0, 1, 0, ...). Since 2 2 spanei : i ∈ N is dense in ` . Let x = (xi) in ` be orthogonal to all ei. Then for any i we have xi = hx, eii = 0 and hence x = 0. P If {en} is a basis for a Hilbert space, then we want to know when k akek converges.

Proposition 2.1. Let M be a closed subspace of H with a countable Hilbert basis en. P∞ 2 Then k=1 akek converges in M if and only if (ak) is in ` . Pn Proof. Let xn = k=1 akek. Then assuming n > m: 2 kxn − xmk = hxn − xm, xn − xmi n m X X = h akek, akeki k=1 k=1 n n X X = h akek, akeki k=m+1 k=m+1 n n X X = ajakhej, eki j=m+1 k=m+1 n X 2 = |aj| . j=m+1

2 Suppose (ak) ∈ ` . Then this last sum converges to 0, implying that (xn) is a Cauchy sequence in M, which must converge to a point x ∈ M.

Conversely, suppose that the sequence (xn) converges in M. Then it is a Cauchy P n 2 sequence, and kxn − xmk → 0, implying that ( j = 1 |aj| ) is a Cauchy sequence in C and must converge as n → ∞.

Theorem 2.2. If a closed subspace M of H has a basis (ek), then we have for x ∈ H: P∞ 2 2 (1) Bessel inequality: |hx, eki| ≤ kxk ; P k=1 (2) P x = khx, ekiek. Pn Proof. Let xn = k=1hx, ekiek. Then xn ∈ M and letting ak = hx, eki we get 2 0 ≤ kx − xnk = hx − xn, x − xni

= hx, xh−hxn, xi − hx, xni + hxn, xni n n 2 X 2 X = kxk − 2 |ak| + ajakhej, eki k=1 j,k=1 n 2 X 2 = kxk − |ak| . k=1

3 Pn 2 2 Hence, k=1 |hx, eki| ≤ kxk , which is an increasing sequence bounded above. Hence the sum on the left must converge and this implies the Bessel inequality.

Pn By the previous proposition the series k=1hx, ekiek converges to a point y ∈ M. Moreover for all m we have n X hx − y, emi = hx, emi − hx, ekihek, emi = 0. k=1 ⊥ Hence x − y ∈ M and so y is the closest point in M to x. An important consequence of the previous theorem is the following result.

Theorem 2.3 (Parseval’s identity). If (ek) is a basis for a Hilbert space H, then for P 2 P∞ 2 all x ∈ X we have x = khx, ekiek and kxk = k=1 |ak| . Proof. Since P x = x for M = H we get the basis expansion for all x ∈ H, and the norm relation follows by the linearity and continuity of the innerproduct. Another consequence is that every separable Hilbert space is isomorphic to `2. 2 Theorem 2.4. Consider the map J : H → ` , x 7→ (hx, eki). Then Bessel’s inequality shows that the Fourier coeﬃcients are in `2. Parseval’s identity also implies that the innerproducts are preserved: hJx, Jyi = hx, yi and so J is an isometry. Hence it is P injective, the map J is also surjective: Since it maps khx, ekiek to (hx, eki), which is a well-deﬁned element in `2.

Department of Mathematics, Norwegian University of Science and Technology, 7491 Trondheim. E-mail address: [email protected]