Hilbert Spaces

Definition. A complex inner product space (or pre-Hilbert space) is a complex vector space X together with an inner product: a function from X × X into C (denoted by hy, xi) satisfying:

(1) (∀ x ∈ X) hx, xi ≥ 0 and hx, xi =0 iff x = 0.

(2) (∀ α, β ∈ C) (∀ x, y, z ∈ X), hz,αx + βyi = αhz, xi + βhz,yi.

(3) (∀ x, y ∈ X) hy, xi = hx, yi

Remarks.

(2) says the inner product is linear in the second variable;

(3) says the inner product is sesquilinear;

(2) and (3) imply hαx + βy,zi =α ¯hx, zi + β¯hy, zi, so the inner product is conjugate linear in the first variable.

Definition. For x ∈ X, let kxk = hx, xi. p Cauchy-Schwarz Inequality. (∀ x, y ∈ X) |hy, xi| ≤ kxk·kyk, with equality iff x and y are linearly dependent. Proof. The result is obvious if hy, xi = 0. Suppose γ ≡ hy, xi= 6 0. Then x =6 0, y =6 0. Let z = γ|γ|−1y. Then hz, xi =γ ¯|γ|−1hy, xi = |γ| > 0. Let v = xkxk−1, w = zkzk−1. Then kvk = kwk = 1 and hw, vi > 0. Since 0 ≤kv − wk2 = hv, vi − 2Rehw, vi + hw,wi, it follows that hw, vi ≤ 1 (with equality iff v = w, which happens iff x and y are linearly dependent). So |hy, xi| = hz, xi = kxk·kzkhw, vi≤kxk·kzk = kxk·kyk. 

Facts.

(1′) (∀ x ∈ X)kxk ≥ 0; kxk =0 iff x = 0.

(2′) (∀ α ∈ C)(∀ x ∈ X) kαxk = |α|·kxk.

(3′) (∀ x, y ∈ X) kx + yk≤kxk + kyk.

83 84 Hilbert Spaces

Proof of (3′):

kx + yk2 = kxk2 +2Rehy, xi + kyk2 ≤kxk2 +2|hy, xi| + kyk2 ≤kxk2 +2kxk·kyk + kyk2.

Hence k·k is a norm on X; called the norm induced by the inner product h·, ·i.

Definition. An inner product space which is complete with respect to the norm induced by the inner product is called a Hilbert space.

n n n Example. X = C . For x =(x1,...,xn) and y =(y1,...,yn) ∈ C , let hy, xi = j=1 yjxj. n 2 2 n P Then kxk = j=1 |xj| is the l -norm on C . qP Examples of Hilbert spaces:

• any finite dimensional inner product space

2 ∞ 2 ∞ • l = {(x1, x2, x3,...) : xk ∈ C, k=1 |xk| < ∞} with hy, xi = k=1 ykxk P P 2 Rn • L (A) for any measurable A ⊂ , with inner product hg, fi = A g(x)f(x)dx. R Incomplete inner product space

b C([a, b]) with hg, fi = a g(x)f(x)dx R C([a, b]) with this inner product is not complete; it is dense in L2([a, b]), which is complete.

Parallelogram Law. Let X be an inner product space. Then (∀ x, y ∈ X)

kx + yk2 + kx − yk2 = 2(kxk2 + kyk2).

Proof. kx + yk2 + kx − yk2 = hx + y, x + yi + hx − y, x − yi = hx, xi + hx, yi + hy, xi + hy,yi + hx, xi−hx, yi−hy, xi + hy,yi = 2(hx, xi + hy,yi)=2(kxk2 + kyk2). 

Polarization Identity. Let X be an inner product space. Then (∀ x, y ∈ X)

1 hy, xi = ky + xk2 −ky − xk2 − iky + ixk2 + iky − ixk2 . 4
Proof. Expanding out the implied inner products, one shows easily that ky + xk2 −ky − xk2 =4Rehy, xi and ky + ixk2 −ky − ixk2 = −4ℑhy, xi.  1 2 2 Note: In a real inner product space, hy, xi = 4 (kx + yk −kx − yk ). Remark. In an inner product space, the inner product determines the norm. The polarization identity shows that the norm determines the inner product. But not every norm on a vector space X is induced by an inner product. Hilbert Spaces 85

Theorem. Suppose (X, k·k) is a normed linear space. The norm k·k is induced by an inner product iff the parallelogram law holds in (X, k·k).

Proof Sketch. (⇒): see above. (⇐): Use the polarization identity to define h·, ·i. Then immediately hx, xi = kxk2, hy, xi = hx, yi, and hy, ixi = ihy, xi. Use the parallelogram law to show hz, x + yi = hz, xi + hz,yi. Then show hy,αxi = αhy, xi successively for α ∈ N, α−1 ∈ N, α ∈ Q, α ∈ R, and finally α ∈ C.  

Continuity of the Inner Product. Let X be an inner product space with induced norm k·k. Then h·, ·i : X × X → C is continuous. Proof. Since X × X and C are metric spaces, it suffices to show sequential continuity. Suppose xn → x and yn → y. Then by the Schwarz inequality,

|hyn, xni−hy, xi| = |hyn, xn − xi + hyn − y, xi|≤kxn − xk·kynk + kxk·kyn − yk→ 0. 

Orthogonality. If hy, xi = 0, we say x and y are orthogonal and write x ⊥ y. For any subset A ⊂ X, define A⊥ = {x ∈ X : hy, xi =0 ∀ y ∈ A}. Since the inner product is linear in the second component and continuous, A⊥ is a closed subspace of X. Also (span(A))⊥ = A⊥, A¯⊥ = A⊥, and (span(A))⊥ = A⊥.

The Pythagorean Theorem. If x1,...,xn ∈ X and xj ⊥ xk for j =6 k, then

n 2 n 2 xj = kxjk .

Xj=1 Xj=1

Proof. If x ⊥ y then kx + yk2 = kxk2 +2Rehy, xi + kyk2 = kxk2 + kyk2. Now use induction. 

Convex Sets. A subset A of a vector space X is called convex if (∀ x, y ∈ A) (∀ t ∈ (0, 1)) (1 − t)x + ty ∈ A.

Examples. (1) Every subspace is convex. (2) In a normed linear space, B(x, ǫ) is convex for ǫ> 0 and x ∈ X. (3) If A is convex and x ∈ X, then A + x ≡{y + x : y ∈ A} is convex.

Theorem. Every nonempty closed convex subset A of a Hilbert space X has a unique element of smallest norm.

1 Proof. Let δ = inf{kxk : x ∈ A}. If x, y ∈ A, then 2 (x + y) ∈ A by convexity, and by the parallelogram law, kx − yk2 = 2(kxk2 + kyk2) −kx + yk2 ≤ 2(kxk2 + kyk2) − 4δ2. 86 Hilbert Spaces

Uniqueness follows: if kxk = kyk = δ, then kx−yk2 ≤ 4δ2 −4δ2 = 0, so x = y. For existence, ∞ choose {yn}n=1 ⊂ A for which kynk→ δ. As n, m →∞,

2 2 2 2 kyn − ymk ≤ 2(kynk + kymk ) − 4δ → 0, so {yn} is Cauchy. By completeness, ∃ y ∈ X for which yn → y, and since A is closed, y ∈ A. Also kyk = lim kynk = δ. 

Corollary. If A is a nonempty closed convex set in a Hilbert space and x ∈ X, then ∃ a unique closest element of A to x.

Proof. Let z be the unique smallest element of the nonempty closed convex set A − x = {y − x : y ∈ A}, and let y = z + x. Then y ∈ A is clearly the unique closest element of A to x. 

Orthogonal Projections onto Closed Subspaces The Projection Theorem. Let M be a closed subspace of a Hilbert space X.

(1) For each x ∈ X, ∃ unique u ∈ M, v ∈ M ⊥ such that x = u + v. (So as vector spaces, X = M ⊕ M ⊥.)

Define the operators P : X → M and Q : X → M ⊥ by P : x 7→ u and Q : x 7→ v.

(2) If x ∈ M, P x = x and Qx = 0; if x ∈ M ⊥, P x = 0 and Qx = x.

(3) P 2 = P , Range(P ) = M, Null Space(P ) = M ⊥; Q2 = Q, Range(Q) = M ⊥, Null Space(Q)= M.

(4) P, Q ∈ B(X,X). kP k = 0 if M = {0}; otherwise kP k = 1. kQk = 0 if M ⊥ = {0}; otherwise kQk = 1.

(5) P x is the unique closest element of M to x, and Qx is the unique closest element of M ⊥ to x.

(6) P + Q = I (obvious by the definition of P and Q).

Proof Sketch. Given x ∈ X, x + M is a closed convex set. Define Qx to be the smallest element of x + M, and let P x = x − Qx. Since Qx ∈ x + M, P x ∈ M. Let z = Qx. Suppose y ∈ M and kyk = 1, and let α = hy, zi. Then z − αy ∈ x + M, so kzk2 ≤ kz − αyk2 = kzk2 − αhz,yi − α¯hy, zi + |α|2 = kzk2 −|α|2. So α = 0. Thus z ∈ M ⊥. Since clearly M ∩ M ⊥ = {0}, the uniqueness of u and v in (1) follows. (2) is immediate from the definition. (3) follows from (1) and (2). For x, y ∈ X, αx+βy =(αP x+βPy)+(αQx+βQy), so by uniqueness in (1), P (αx + βy)= αP x + βPy and Q(αx + βy)= αQx + βQy. By the Pythagorean Theorem, kxk2 = kP xk2 + kQxk2, so P, Q ∈ B(X,X) and kP k, kQk ≤ 1. The rest of (4) follows from (2). Fix x ∈ X. If y ∈ X, then kx−yk2 = kP x−Pyk2 +kQx−Qyk2. Hilbert Spaces 87

If y ∈ M, then kx − yk2 = kP x − yk2 + kQxk2, which is clearly minimized by taking y = P x. If y ∈ M ⊥, then kx−yk2 = kP xk2 +kQx−yk2, which is clearly minimized by taking y = Qx. 

Corollary. If M is a closed subspace of a Hilbert space X, then (M ⊥)⊥ = M. In general, for any A ⊂ X,(A⊥)⊥ = span{A}, which is the smallest closed subspace of X containing A, often called the closed linear span of A.

Bounded Linear Functionals and Riesz Representation Theorem

Proposition. Let X be an inner product space, fix y ∈ X, and define fy : X → C by ∗ fy(x)= hy, xi. Then fy ∈ X and kfyk = kyk.

∗ Proof. |fy(x)| = |hy, xi|≤ kxk·kyk, so fy ∈ X and kfyk≤kyk. Since |fy(y)| = |hy,yi| = 2 kyk , kfyk≥kyk. So kfyk = kyk. 

Theorem. Let X be a Hilbert space.

∗ (1) If f ∈ X , then ∃ a unique y ∈ X ∋ f = fy, i.e., f(x)= hy, xi ∀ x ∈ X.

∗ (2) The map ψ : X → X given by ψ : y 7→ fy is a conjugate linear isometry of X onto X∗.

Proof.

(1) If f ≡ 0, let y = 0. If f ∈ X∗ and f 6≡ 0, then M ≡ f −1({0}) is a proper closed subspace of X, so ∃ z ∈ M ⊥ ∋ kzk = 1. Let α = f(z) and y = αz. Given x ∈ X, u ≡ f(x)z − f(z)x ∈ M, so0= hz,ui = f(x)hz, zi − f(z)hz, xi = f(x) −hαz,xi = f(x) −hy, xi, i.e., f(x)= hy, xi. Uniqueness: if hy1, xi = hy2, xi ∀ x ∈ X, then (letting 2 x = y1 − y2) ky1 − y2k = 0, so y1 = y2.

(2) follows immediately from (1), the previous proposition, and the conjugate linearity of the inner product in the first variable. 

Corollary. X∗ is a Hilbert space with the inner product hg, fi = hψ−1(g), ψ−1(f)i (i.e., hfy, fxi = hy, xi = hx, yi).

Proof. Clearly hf, fi ≥ 0, hf, fi = 0 iff ψ−1(f) = 0iff f = 0, and hg, fi = hf,gi.

Also hfy, α1fx1 + α2fx2 i = hfy, fα¯1x1+¯α2x2 i = hy, α¯1x1 +α ¯2x2i = α1hy, x1i + α2hy, x2i = ∗ 2 α1hfy, fx1 i + α2hfy, fx2 i, so h·, ·i is an inner product on X . Since hfy, fyi = hy,yi = kyk = 2 ∗ ∗ kfyk , h·, ·i induces the norm on X . Since X is complete, it is a Hilbert space. 

Remark. Part (1) of the Theorem above is often called [one of] the Riesz Representation Theorem[s]. 88 Hilbert Spaces

Strong convergence/Weak convergence

Let X be a Hilbert space. We say xn → x strongly if kxn − xk → 0 as n → ∞. This is the usual concept of convergence in the metric induced by the norm, and is also called convergence in norm. We say xn → x weakly if (∀ y ∈ X) hy, xni→hy, xi as n → ∞. w (Other common notations for weak convergence are xn ⇀ x, xn → x.) The Cauchy-Schwarz inequality shows that strong convergence implies weak convergence. Also, if xn → x strongly, then kxnk→kxk since |kxnk−kxk|≤kxn − xk. Example. (Weak convergence does not imply strong convergence if dim X = ∞). Let ւ kth entry 2 X = l . For k = 1, 2,..., let ek = (0,..., 0, 1, 0,...) (so {ek : k = 1, 2,...} is an orthonormal set in l2).

Claim. ek → 0 weakly as k →∞.

2 ∞ 2  Proof. Fix y ∈ l . Then k=1 |yk| < ∞, so yk → 0. So hy, eki = yk → 0. P Note that kekk = 1, so ek does not converge to zero strongly. Remark. If dim X < ∞, then weak convergence ⇒ strong convergence (exercise).

Theorem. Suppose xn → x weakly in a Hilbert space X. Then

(a) kxk ≤ lim infk→∞ kxkk

(b) If kxkk→kxk, then xk → x strongly (i.e., kxk − xk→ 0).

Proof.

2 2 2 2 (a) 0 ≤ kx − xkk = kxk − 2Rehx, xki + kxkk . By hypothesis, hx, xki→hx, xi = kxk . 2 2 2 2 2 So taking lim inf above, 0 ≤kxk − 2kxk + lim inf kxkk , i.e. kxk ≤ lim inf kxkk .

2 2 2 (b) If xk → x weakly and kxkk→kxk, then kx − xkk = kxk − 2Rehxk, xi + kxkk → kxk2 − 2kxk2 + kxk2 = 0. 

Remark. The Uniform Boundedness Principle implies that if xk → x weakly, then kxkk is bounded.

Orthogonal Sets

Definition. Let X be an inner product space. Let A be a set (not necessarily countable). A set {uα}α∈A ⊂ X is called an orthogonal set if (∀ α =6 β ∈ A) huβ,uαi = 0. (Often it is also assumed that each uα =6 0.) Hilbert Spaces 89

Orthonormal Sets

Definition. Let X be an inner product space. A set {uα}α∈A is called an orthonormal set if it is orthogonal and (∀ α ∈ A) kuαk = 1. For each x ∈ X, define a function x : A → C by x(α) = huα, xi. The x(α)’s are called the Fourier coefficients of x with respect to the b orthonormal set {uα}α∈A. b b

Theorem. If {u1,...,uk} is an orthonormal set in an inner product space X, and x = k 2 l 2 j=1 cjuj, then cj = huj, xi for 1 ≤ j ≤ k and kxk = j=1 |cj| P P Proof. hui, xi = cjhui,uji = ci. Now use the Pythagorean Theorem.  P Corollary. Every orthonormal set is linearly independent.

Example. If A is finite, say A = {1, 2,...,n}, then for any x ∈ X, we know that the closest n element of span{u1,...,un} to x is k=1huk, xiuk. P Theorem. (Gram-Schmidt process) Let V be a subspace of an inner product space X, and suppose V has a finite or countable basis {xn}n≥1. Then V has a basis {un}n≥1 which is orthonormal (we reserve the term “orthonormal basis” to mean something else); moreover we can choose {un}n≥1 so that for all m ≥ 1, span{u1,...,um} = span{x1,...,xm}.

x1 u u1 u1,...,u 1 Proof Sketch.. Define { n} inductively. Start with = kx1k . Having defined n− , 1 let v = x − n− hu , x iu and u = vn .  n n j=1 j n j n kvnk P

Theorem. Let V be a finite dimensional subspace of a Hilbert space X. Let {u1,...,un} be a basis for V which is orthonormal, and let P be the orthogonal projection of X onto V . n 2 2 2 n 2 2 Then P x = j=1huj, xiuj and kxk = kP xk + kQxk = j=1 |huj, xi| + kQxk . P P Definition. Let A be a nonempty set. For each α ∈ A, let yα be a nonnegative real number.

Define α∈A yα = sup{ α∈F yα : F ⊂ A and F is finite}. P P Proposition. If α∈A yα < ∞, then yα =6 0 for at most countably many α. P −1 Proof. For each k, it is clear that Ak ≡{α : yα >k } is a finite set. But {α : yα =6 0} = ∞  ∪k=1Ak.

Definition. Let A be a nonempty set. Define l2(A) to be the set of functions f : A → C 2 2 for which α∈A |f(α)| < ∞. Then l (A) is a Hilbert space with inner product hg, fi =

α∈A g(α)Pf(α) and norm kfk2 = hf, fi. P p Bessel’s Inequality. Let {uα}α∈A be an orthonormal set in a Hilbert space X, let x ∈ X, 2 2 and let x(α)= huα, xi. Then α∈A |x(α)| ≤kxk . P Proof. bBy the previous Theorem, thisb is true for every finite subset of A. Take the sup. 

Corollary. Let {uα}α∈A, x be as above. Then 90 Hilbert Spaces

2 (1) x ∈ l (A) and kxk2 ≤kxk

(2) {bα ∈ A : x(α) =06 b } is countable.

b Theorem. Let X be a Hilbert space and let {uα}α∈A be an orthonormal set. Define 2 F : X → l (A) (F is for Fourier) by F (x)= x where x(α)= huα, xi. Then F is a bounded linear operator with kF k = 1, which maps X onto l2(A). b b Proof. Clearly F is linear. By (1) of the Corollary, F is bounded and kF k ≤ 1. If x = uα for 2 some α ∈ A, kxk2 =1= kxk, so kF k = 1. Given f ∈ l (A), f(α) =6 0 only for a countable k set Af ⊂ A; enumerate them α1, α2, α3,.... Let xk = j=1 f(αj)uj. Clearly xk(α) = f(α) b for α1,...,αk and xk(α) = 0 otherwise. So P b ∞ b 2 2 kxk − fk2 = |f(αj)| → 0 as k →∞. j=Xk+1 b 2 2 Thus xk → f in l (A), and in particular xk is a Cauchy sequence in l (A). Since each xk is a finite linear combination of the uα’s, kxj − xkk = kxj − xkk2, so {xk} is Cauchy in X, so b b xk → x in X for some x ∈ X. For each α ∈ A, b b x(α)= huα, xi = lim huα, xki = lim xk(α)= f(α). k→∞ k→∞ So F (x)= f and F isb onto. b 

Theorem. Let X be a Hilbert space. Every orthonormal set in X is contained in a maximal orthonormal set (i.e., an orthonormal set not properly contained in any orthonormal set).

Proof. Zorn’s lemma. 

Corollary. Every Hilbert space has a maximal orthonormal set.

Theorem. Let {uα}α∈A be an orthonormal set in a Hilbert space X. The following condi- tions are equivalent:

(a) {uα}α∈A is a maximal orthonormal set.

(b) The set of finite linear combinations of the uα’s is dense in X.

2 2 (c) (∀ x ∈ X) kxk = α∈A |x(α)| (Parseval’s relation). P (d) (∀ x, y ∈ X) hy, xi = α∈bA y(α)x(α). P (e) (∀ x ∈ X) if (∀ α ∈ A) huα, xbi =b 0 then x = 0.

Proof.

(a) ⇒ (b): Let V = span{uα : α ∈ A} and M = V¯ . Then M is a closed subspace. Since ⊥ ⊥ {uα} is maximal, V = {0}, so M = {0}, so M = X. Hilbert Spaces 91

(b) ⇒ (c): Clear if x = 0. Given x =6 0, and given ǫ> 0 (WLOG assume ǫ< kxk), choose

y ∈ V ∋kx − yk < ǫ, say y ∈ span{uα1 ,...,uαk }. Let z = x(α1)uα1 + ··· + x(αk)uαk .

Then z minimizes kx − wk over w ∈ span{uα1 ,...,uαk } so kx − zk≤kx − yk < ǫ. 2 2 2 k b 2 b 2 Thus kxk < kzk + ǫ, so (kxk − ǫ) < kzk and kzk = j=1 |x(αj)| ≤ α∈A |x(α)| . Let ǫ → 0 to get kxk2 ≤ |x(α)|2. The other inequalityP is Bessel’s inequality.P α∈A b b P (c) ⇒ (d): Use polarization. Only countablyb many terms in the sum are nonzero. 2 (d) ⇒ (e): Suppose (∀ α ∈ A) hx, uαi = 0. Then x(α) ≡ 0, so kxk = hx, xi = 0, so x = 0.  (e) ⇒ (a): If {uα} is not maximal, then ∃ x =06 ∋hb x, uαi = 0 for all α ∈ A.

Definition. An orthonormal set {uα} in a Hilbert space X satisfying the conditions in the previous theorem is called a complete orthonormal set (or a complete orthonormal system) or an orthonormal basis in X. Caution. If X is infinite dimensional, an orthonormal basis is not a basis in the usual definition of a basis for a vector space (i.e., each x ∈ X has a unique representation as a finite linear combination of basis elements). Such a basis in this context is called a Hamel basis. Definition. Let X and Y be inner product spaces. A map T : X → Y which is linear, bijective, and preserves inner products (i.e., (∀ x, y ∈ X) hx, yi = hTx,Tyi — this implies T is an isometry kxk = kT xk) is called a unitary isomorphism.

Corollary. If X is a Hilbert space and {uα}α∈A is an orthonormal basis of X, then the map 2 F : X → l (A) mapping x 7→ x (where x(α)= hx, uαi) is a unitary isomorphism. 2 Corollary. Every Hilbert spaceb is unitarilyb isomorphic to l (A) for some A.

Norm Convergence of Fourier Series

Theorem. Let X be a Hilbert space, {uα}α∈A be an orthonormal set in X, and let x ∈ X. 2 2 Let {αj}j≥1 be any enumeration of {α ∈ A : huα, xi=0 6 }. Then kxk = j≥1 |huαj , xi| (i.e. n P Parseval’s Equality holds for this x) iff limn→∞ x − j=1huαj , xiuαj = 0 (i.e. the Fourier ∞ series x(α )u converges to x in norm). P j=1 j αj P Proof. Let bMn = span{uα1 ,...,uαn } and let Pn be the orthogonal projection onto Mn (so ⊥ n 2 I − Pn is the orthogonal projection onto Mn ). Then Pnx = j=1huαj , xiuαj and kPnxk = n 2 2 2 2 2 2 2 j=1 |huαj , xi| . Also kxk = kPnxk + k(I − Pn)xk , so kxk P−kPnxk = kx − Pnxk . Hence 2 2 2 2 2 Pkxk = j≥1 |huαj , xi| iff limn→∞ kPnxk = kxk iff limn→∞ kx − Pnxk = 0, which is the desired conclusion.P (Note: If {α ∈ A : huα, xi=0 6 } is finite, say {α1,...,αn}, then Parseval 2 2 n holds iff kPnxk = kxk iff x = Pnx, i.e., x = j=1huαj , xiuαj ∈ Mn.) P Corollary. Let {uα}α∈A be an orthonormal set in a Hilbert space X. Then {uα} is an orthonormal basis iff for each x ∈ X and each enumeration {αj}j≥1 of {α ∈ A : huα, xi=0 6 }, n limn→∞ kx − j=1huαj , xiuαj k = 0. P 92 Hilbert Spaces

Cardinality of Orthonormal Bases

Proposition. l2(A) is unitarily isomorphic to l2(B) iff card(A) = card(B). Proposition. Any pair of orthonormal bases in a Hilbert space have the same cardinality. Proposition. A Hilbert space X is separable iff it has a countable orthonormal basis. Remark. For a separable Hilbert space X, one can show directly without invoking Zorn’s lemma that X has a countable complete orthonormal set.

Proof. Clear if dim X < ∞. Suppose dim X = ∞. Let z1, z2,... be a countable dense subset. Apply Gram-Schmidt (dropping zero vectors along the way) to get an orthonormal sequence u1,u2,... whose finite linear combinations include z1,z,..., and thus are dense.  Theorem. (Orthogonal projection in terms of orthonormal bases.) Let X be a Hilbert space, and let M be a closed subspace of X. Let {vβ}β∈B be a complete orthonormal set in ⊥ M, and let {wγ}γ∈C be a complete orthonormal set in M . Then {vβ}∪{wγ} is a complete orthonormal set in X. The orthogonal projection of X onto M is P x = β∈Bhvβ, xivβ, and ⊥ the orthogonal projection of X onto M is Qx = γ∈C hwγ, xiwγ. P P Proof. Follows directly from X = M ⊕ M ⊥ and the projection theorem.  Example. (Orthogonal Polynomials in weighted L2 spaces.) Fix a, b ∈ R with −∞ 0 on (a, b) and a w(x)dx < ∞. The function w is called the weight function. For example, take w(x)=(1R− x2)−1/2 on (−1, 1). Define b 2 2 Lw(a, b)= f : f is measurable on (a, b) and |f(x)| w(x)dx < ∞  Za  b 2 and define hg, fiw = a g(x)f(x)w(x)dx for f,g ∈ Lw(a, b). Then (after identifying f and g 2 when f = g a.e.), LwR(a, b) is a Hilbert space. 2 Claim. Polynomials are dense in Lw(a, b).

∞ 2 b 2 Proof. First note that if f ∈ L (a, b), then f ∈ Lw(a, b) since |f(x)| w(x)dx ≤ a 1 2 b b R 2 kfk∞ a w(x)dx, and thus kfkw ≤ Mkfk∞, where M = a w(x)dx < ∞. Given R2 1 R  f ∈ Lw(a, b), ∃ g ∈ C[a, b] for which kf − gkw < 2 ǫ (exercise). By the Weierstrass Approx- imation Theorem, polynomials are dense in (C[a, b], k·k∞), so ∃ a polynomial p for which −1 1 1 1 kg −pk∞ < (2M) ǫ. Then kf −pkw ≤kf −gkw +kg −pkw < 2 ǫ+Mkg −pk∞ < 2 ǫ+ 2 ǫ = ǫ.  Applying the Gram-Schmidt process to the linearly independent set {1, x, x2,...} using the 2 inner product of Lw(a, b) produces a sequence of polynomials which are orthonormal in 2 Lw(a, b). 2 Theorem. The orthogonal polynomials in Lw(a, b) are a complete orthonormal set in 2 Lw(a, b). Proof. Finite linear combinations are dense.