<<

Hilbert Spaces

Deﬁnition. A complex inner product (or pre-) is a complex X together with an inner product: a function from X × X into C (denoted by hy, xi) satisfying:

(1) (∀ x ∈ X) hx, xi ≥ 0 and hx, xi =0 iﬀ x = 0.

(2) (∀ α, β ∈ C) (∀ x, y, z ∈ X), hz,αx + βyi = αhz, xi + βhz,yi.

(3) (∀ x, y ∈ X) hy, xi = hx, yi

Remarks.

(2) says the inner product is linear in the second variable;

(3) says the inner product is sesquilinear;

(2) and (3) imply hαx + βy,zi =α ¯hx, zi + β¯hy, zi, so the inner product is conjugate linear in the ﬁrst variable.

Deﬁnition. For x ∈ X, let kxk = hx, xi. p Cauchy-Schwarz Inequality. (∀ x, y ∈ X) |hy, xi| ≤ kxk·kyk, with equality iﬀ x and y are linearly dependent. Proof. The result is obvious if hy, xi = 0. Suppose γ ≡ hy, xi= 6 0. Then x =6 0, y =6 0. Let z = γ|γ|−1y. Then hz, xi =γ ¯|γ|−1hy, xi = |γ| > 0. Let v = xkxk−1, w = zkzk−1. Then kvk = kwk = 1 and hw, vi > 0. Since 0 ≤kv − wk2 = hv, vi − 2Rehw, vi + hw,wi, it follows that hw, vi ≤ 1 (with equality iﬀ v = w, which happens iﬀ x and y are linearly dependent). So |hy, xi| = hz, xi = kxk·kzkhw, vi≤kxk·kzk = kxk·kyk. 

Facts.

(1′) (∀ x ∈ X)kxk ≥ 0; kxk =0 iﬀ x = 0.

(2′) (∀ α ∈ C)(∀ x ∈ X) kαxk = |α|·kxk.

(3′) (∀ x, y ∈ X) kx + yk≤kxk + kyk.

83 84 Hilbert Spaces

Proof of (3′):

kx + yk2 = kxk2 +2Rehy, xi + kyk2 ≤kxk2 +2|hy, xi| + kyk2 ≤kxk2 +2kxk·kyk + kyk2.

Hence k·k is a on X; called the norm induced by the inner product h·, ·i.

Deﬁnition. An which is complete with respect to the norm induced by the inner product is called a Hilbert space.

n n n Example. X = C . For x =(x1,...,xn) and y =(y1,...,yn) ∈ C , let hy, xi = j=1 yjxj. n 2 2 n P Then kxk = j=1 |xj| is the l -norm on C . qP Examples of Hilbert spaces:

• any ﬁnite dimensional inner product space

2 ∞ 2 ∞ • l = {(x1, x2, x3,...) : xk ∈ C, k=1 |xk| < ∞} with hy, xi = k=1 ykxk P P 2 Rn • L (A) for any measurable A ⊂ , with inner product hg, fi = A g(x)f(x)dx. R Incomplete inner product space

b C([a, b]) with hg, fi = a g(x)f(x)dx R C([a, b]) with this inner product is not complete; it is dense in L2([a, b]), which is complete.

Parallelogram Law. Let X be an inner product space. Then (∀ x, y ∈ X)

kx + yk2 + kx − yk2 = 2(kxk2 + kyk2).

Proof. kx + yk2 + kx − yk2 = hx + y, x + yi + hx − y, x − yi = hx, xi + hx, yi + hy, xi + hy,yi + hx, xi−hx, yi−hy, xi + hy,yi = 2(hx, xi + hy,yi)=2(kxk2 + kyk2). 

Polarization Identity. Let X be an inner product space. Then (∀ x, y ∈ X)

1 hy, xi = ky + xk2 −ky − xk2 − iky + ixk2 + iky − ixk2 . 4  Proof. Expanding out the implied inner products, one shows easily that ky + xk2 −ky − xk2 =4Rehy, xi and ky + ixk2 −ky − ixk2 = −4ℑhy, xi.  1 2 2 Note: In a real inner product space, hy, xi = 4 (kx + yk −kx − yk ). Remark. In an inner product space, the inner product determines the norm. The shows that the norm determines the inner product. But not every norm on a vector space X is induced by an inner product. Hilbert Spaces 85

Theorem. Suppose (X, k·k) is a normed linear space. The norm k·k is induced by an inner product iﬀ the holds in (X, k·k).

Proof Sketch. (⇒): see above. (⇐): Use the polarization identity to deﬁne h·, ·i. Then immediately hx, xi = kxk2, hy, xi = hx, yi, and hy, ixi = ihy, xi. Use the parallelogram law to show hz, x + yi = hz, xi + hz,yi. Then show hy,αxi = αhy, xi successively for α ∈ N, α−1 ∈ N, α ∈ Q, α ∈ R, and ﬁnally α ∈ C.  

Continuity of the Inner Product. Let X be an inner product space with induced norm k·k. Then h·, ·i : X × X → C is continuous. Proof. Since X × X and C are metric spaces, it suﬃces to show sequential continuity. Suppose xn → x and yn → y. Then by the Schwarz inequality,

|hyn, xni−hy, xi| = |hyn, xn − xi + hyn − y, xi|≤kxn − xk·kynk + kxk·kyn − yk→ 0. 

Orthogonality. If hy, xi = 0, we say x and y are orthogonal and write x ⊥ y. For any subset A ⊂ X, deﬁne A⊥ = {x ∈ X : hy, xi =0 ∀ y ∈ A}. Since the inner product is linear in the second component and continuous, A⊥ is a closed subspace of X. Also (span(A))⊥ = A⊥, A¯⊥ = A⊥, and (span(A))⊥ = A⊥.

The . If x1,...,xn ∈ X and xj ⊥ xk for j =6 k, then

n 2 n 2 xj = kxjk .

Xj=1 Xj=1

Proof. If x ⊥ y then kx + yk2 = kxk2 +2Rehy, xi + kyk2 = kxk2 + kyk2. Now use induction. 

Convex Sets. A subset A of a vector space X is called convex if (∀ x, y ∈ A) (∀ t ∈ (0, 1)) (1 − t)x + ty ∈ A.

Examples. (1) Every subspace is convex. (2) In a normed linear space, B(x, ǫ) is convex for ǫ> 0 and x ∈ X. (3) If A is convex and x ∈ X, then A + x ≡{y + x : y ∈ A} is convex.

Theorem. Every nonempty closed convex subset A of a Hilbert space X has a unique element of smallest norm.

1 Proof. Let δ = inf{kxk : x ∈ A}. If x, y ∈ A, then 2 (x + y) ∈ A by convexity, and by the parallelogram law, kx − yk2 = 2(kxk2 + kyk2) −kx + yk2 ≤ 2(kxk2 + kyk2) − 4δ2. 86 Hilbert Spaces

Uniqueness follows: if kxk = kyk = δ, then kx−yk2 ≤ 4δ2 −4δ2 = 0, so x = y. For existence, ∞ choose {yn}n=1 ⊂ A for which kynk→ δ. As n, m →∞,

2 2 2 2 kyn − ymk ≤ 2(kynk + kymk ) − 4δ → 0, so {yn} is Cauchy. By completeness, ∃ y ∈ X for which yn → y, and since A is closed, y ∈ A. Also kyk = lim kynk = δ. 

Corollary. If A is a nonempty closed convex in a Hilbert space and x ∈ X, then ∃ a unique closest element of A to x.

Proof. Let z be the unique smallest element of the nonempty closed A − x = {y − x : y ∈ A}, and let y = z + x. Then y ∈ A is clearly the unique closest element of A to x. 

Orthogonal Projections onto Closed Subspaces The Projection Theorem. Let M be a closed subspace of a Hilbert space X.

(1) For each x ∈ X, ∃ unique u ∈ M, v ∈ M ⊥ such that x = u + v. (So as vector spaces, X = M ⊕ M ⊥.)

Deﬁne the operators P : X → M and Q : X → M ⊥ by P : x 7→ u and Q : x 7→ v.

(2) If x ∈ M, P x = x and Qx = 0; if x ∈ M ⊥, P x = 0 and Qx = x.

(3) P 2 = P , Range(P ) = M, Null Space(P ) = M ⊥; Q2 = Q, Range(Q) = M ⊥, Null Space(Q)= M.

(4) P, Q ∈ B(X,X). kP k = 0 if M = {0}; otherwise kP k = 1. kQk = 0 if M ⊥ = {0}; otherwise kQk = 1.

(5) P x is the unique closest element of M to x, and Qx is the unique closest element of M ⊥ to x.

(6) P + Q = I (obvious by the deﬁnition of P and Q).

Proof Sketch. Given x ∈ X, x + M is a closed convex set. Deﬁne Qx to be the smallest element of x + M, and let P x = x − Qx. Since Qx ∈ x + M, P x ∈ M. Let z = Qx. Suppose y ∈ M and kyk = 1, and let α = hy, zi. Then z − αy ∈ x + M, so kzk2 ≤ kz − αyk2 = kzk2 − αhz,yi − α¯hy, zi + |α|2 = kzk2 −|α|2. So α = 0. Thus z ∈ M ⊥. Since clearly M ∩ M ⊥ = {0}, the uniqueness of u and v in (1) follows. (2) is immediate from the deﬁnition. (3) follows from (1) and (2). For x, y ∈ X, αx+βy =(αP x+βPy)+(αQx+βQy), so by uniqueness in (1), P (αx + βy)= αP x + βPy and Q(αx + βy)= αQx + βQy. By the Pythagorean Theorem, kxk2 = kP xk2 + kQxk2, so P, Q ∈ B(X,X) and kP k, kQk ≤ 1. The rest of (4) follows from (2). Fix x ∈ X. If y ∈ X, then kx−yk2 = kP x−Pyk2 +kQx−Qyk2. Hilbert Spaces 87

If y ∈ M, then kx − yk2 = kP x − yk2 + kQxk2, which is clearly minimized by taking y = P x. If y ∈ M ⊥, then kx−yk2 = kP xk2 +kQx−yk2, which is clearly minimized by taking y = Qx. 

Corollary. If M is a closed subspace of a Hilbert space X, then (M ⊥)⊥ = M. In general, for any A ⊂ X,(A⊥)⊥ = span{A}, which is the smallest closed subspace of X containing A, often called the closed of A.

Bounded Linear Functionals and Riesz

Proposition. Let X be an inner product space, ﬁx y ∈ X, and deﬁne fy : X → C by ∗ fy(x)= hy, xi. Then fy ∈ X and kfyk = kyk.

∗ Proof. |fy(x)| = |hy, xi|≤ kxk·kyk, so fy ∈ X and kfyk≤kyk. Since |fy(y)| = |hy,yi| = 2 kyk , kfyk≥kyk. So kfyk = kyk. 

Theorem. Let X be a Hilbert space.

∗ (1) If f ∈ X , then ∃ a unique y ∈ X ∋ f = fy, i.e., f(x)= hy, xi ∀ x ∈ X.

∗ (2) The map ψ : X → X given by ψ : y 7→ fy is a conjugate linear of X onto X∗.

Proof.

(1) If f ≡ 0, let y = 0. If f ∈ X∗ and f 6≡ 0, then M ≡ f −1({0}) is a proper closed subspace of X, so ∃ z ∈ M ⊥ ∋ kzk = 1. Let α = f(z) and y = αz. Given x ∈ X, u ≡ f(x)z − f(z)x ∈ M, so0= hz,ui = f(x)hz, zi − f(z)hz, xi = f(x) −hαz,xi = f(x) −hy, xi, i.e., f(x)= hy, xi. Uniqueness: if hy1, xi = hy2, xi ∀ x ∈ X, then (letting 2 x = y1 − y2) ky1 − y2k = 0, so y1 = y2.

(2) follows immediately from (1), the previous proposition, and the conjugate linearity of the inner product in the ﬁrst variable. 

Corollary. X∗ is a Hilbert space with the inner product hg, fi = hψ−1(g), ψ−1(f)i (i.e., hfy, fxi = hy, xi = hx, yi).

Proof. Clearly hf, fi ≥ 0, hf, fi = 0 iﬀ ψ−1(f) = 0iﬀ f = 0, and hg, fi = hf,gi.

Also hfy, α1fx1 + α2fx2 i = hfy, fα¯1x1+¯α2x2 i = hy, α¯1x1 +α ¯2x2i = α1hy, x1i + α2hy, x2i = ∗ 2 α1hfy, fx1 i + α2hfy, fx2 i, so h·, ·i is an inner product on X . Since hfy, fyi = hy,yi = kyk = 2 ∗ ∗ kfyk , h·, ·i induces the norm on X . Since X is complete, it is a Hilbert space. 

Remark. Part (1) of the Theorem above is often called [one of] the Riesz Representation Theorem[s]. 88 Hilbert Spaces

Strong convergence/Weak convergence

Let X be a Hilbert space. We say xn → x strongly if kxn − xk → 0 as n → ∞. This is the usual concept of convergence in the metric induced by the norm, and is also called convergence in norm. We say xn → x weakly if (∀ y ∈ X) hy, xni→hy, xi as n → ∞. w (Other common notations for weak convergence are xn ⇀ x, xn → x.) The Cauchy-Schwarz inequality shows that strong convergence implies weak convergence. Also, if xn → x strongly, then kxnk→kxk since |kxnk−kxk|≤kxn − xk. Example. (Weak convergence does not imply strong convergence if dim X = ∞). Let ւ kth entry 2 X = l . For k = 1, 2,..., let ek = (0,..., 0, 1, 0,...) (so {ek : k = 1, 2,...} is an orthonormal set in l2).

Claim. ek → 0 weakly as k →∞.

2 ∞ 2  Proof. Fix y ∈ l . Then k=1 |yk| < ∞, so yk → 0. So hy, eki = yk → 0. P Note that kekk = 1, so ek does not converge to zero strongly. Remark. If dim X < ∞, then weak convergence ⇒ strong convergence (exercise).

Theorem. Suppose xn → x weakly in a Hilbert space X. Then

(a) kxk ≤ lim infk→∞ kxkk

(b) If kxkk→kxk, then xk → x strongly (i.e., kxk − xk→ 0).

Proof.

2 2 2 2 (a) 0 ≤ kx − xkk = kxk − 2Rehx, xki + kxkk . By hypothesis, hx, xki→hx, xi = kxk . 2 2 2 2 2 So taking lim inf above, 0 ≤kxk − 2kxk + lim inf kxkk , i.e. kxk ≤ lim inf kxkk .

2 2 2 (b) If xk → x weakly and kxkk→kxk, then kx − xkk = kxk − 2Rehxk, xi + kxkk → kxk2 − 2kxk2 + kxk2 = 0. 

Remark. The Uniform Boundedness Principle implies that if xk → x weakly, then kxkk is bounded.

Orthogonal Sets

Deﬁnition. Let X be an inner product space. Let A be a set (not necessarily countable). A set {uα}α∈A ⊂ X is called an orthogonal set if (∀ α =6 β ∈ A) huβ,uαi = 0. (Often it is also assumed that each uα =6 0.) Hilbert Spaces 89

Orthonormal Sets

Deﬁnition. Let X be an inner product space. A set {uα}α∈A is called an orthonormal set if it is orthogonal and (∀ α ∈ A) kuαk = 1. For each x ∈ X, deﬁne a function x : A → C by x(α) = huα, xi. The x(α)’s are called the Fourier coeﬃcients of x with respect to the b orthonormal set {uα}α∈A. b b

Theorem. If {u1,...,uk} is an orthonormal set in an inner product space X, and x = k 2 l 2 j=1 cjuj, then cj = huj, xi for 1 ≤ j ≤ k and kxk = j=1 |cj| P P Proof. hui, xi = cjhui,uji = ci. Now use the Pythagorean Theorem.  P Corollary. Every orthonormal set is linearly independent.

Example. If A is ﬁnite, say A = {1, 2,...,n}, then for any x ∈ X, we know that the closest n element of span{u1,...,un} to x is k=1huk, xiuk. P Theorem. (Gram-Schmidt process) Let V be a subspace of an inner product space X, and suppose V has a ﬁnite or countable {xn}n≥1. Then V has a basis {un}n≥1 which is orthonormal (we reserve the term “” to mean something else); moreover we can choose {un}n≥1 so that for all m ≥ 1, span{u1,...,um} = span{x1,...,xm}.

x1 u u1 u1,...,u 1 Proof Sketch.. Deﬁne { n} inductively. Start with = kx1k . Having deﬁned n− , 1 let v = x − n− hu , x iu and u = vn .  n n j=1 j n j n kvnk P

Theorem. Let V be a ﬁnite dimensional subspace of a Hilbert space X. Let {u1,...,un} be a basis for V which is orthonormal, and let P be the orthogonal projection of X onto V . n 2 2 2 n 2 2 Then P x = j=1huj, xiuj and kxk = kP xk + kQxk = j=1 |huj, xi| + kQxk . P P Deﬁnition. Let A be a nonempty set. For each α ∈ A, let yα be a nonnegative .

Deﬁne α∈A yα = sup{ α∈F yα : F ⊂ A and F is ﬁnite}. P P Proposition. If α∈A yα < ∞, then yα =6 0 for at most countably many α. P −1 Proof. For each k, it is clear that Ak ≡{α : yα >k } is a ﬁnite set. But {α : yα =6 0} = ∞  ∪k=1Ak.

Deﬁnition. Let A be a nonempty set. Deﬁne l2(A) to be the set of functions f : A → C 2 2 for which α∈A |f(α)| < ∞. Then l (A) is a Hilbert space with inner product hg, fi =

α∈A g(α)Pf(α) and norm kfk2 = hf, fi. P p Bessel’s Inequality. Let {uα}α∈A be an orthonormal set in a Hilbert space X, let x ∈ X, 2 2 and let x(α)= huα, xi. Then α∈A |x(α)| ≤kxk . P Proof. bBy the previous Theorem, thisb is true for every ﬁnite subset of A. Take the sup. 

Corollary. Let {uα}α∈A, x be as above. Then 90 Hilbert Spaces

2 (1) x ∈ l (A) and kxk2 ≤kxk

(2) {bα ∈ A : x(α) =06 b } is countable.

b Theorem. Let X be a Hilbert space and let {uα}α∈A be an orthonormal set. Deﬁne 2 F : X → l (A) (F is for Fourier) by F (x)= x where x(α)= huα, xi. Then F is a bounded linear with kF k = 1, which maps X onto l2(A). b b Proof. Clearly F is linear. By (1) of the Corollary, F is bounded and kF k ≤ 1. If x = uα for 2 some α ∈ A, kxk2 =1= kxk, so kF k = 1. Given f ∈ l (A), f(α) =6 0 only for a countable k set Af ⊂ A; enumerate them α1, α2, α3,.... Let xk = j=1 f(αj)uj. Clearly xk(α) = f(α) b for α1,...,αk and xk(α) = 0 otherwise. So P b ∞ b 2 2 kxk − fk2 = |f(αj)| → 0 as k →∞. j=Xk+1 b 2 2 Thus xk → f in l (A), and in particular xk is a Cauchy in l (A). Since each xk is a ﬁnite of the uα’s, kxj − xkk = kxj − xkk2, so {xk} is Cauchy in X, so b b xk → x in X for some x ∈ X. For each α ∈ A, b b x(α)= huα, xi = lim huα, xki = lim xk(α)= f(α). k→∞ k→∞ So F (x)= f and F isb onto. b 

Theorem. Let X be a Hilbert space. Every orthonormal set in X is contained in a maximal orthonormal set (i.e., an orthonormal set not properly contained in any orthonormal set).

Proof. Zorn’s lemma. 

Corollary. Every Hilbert space has a maximal orthonormal set.

Theorem. Let {uα}α∈A be an orthonormal set in a Hilbert space X. The following condi- tions are equivalent:

(a) {uα}α∈A is a maximal orthonormal set.

(b) The set of ﬁnite linear combinations of the uα’s is dense in X.

2 2 (c) (∀ x ∈ X) kxk = α∈A |x(α)| (Parseval’s relation). P (d) (∀ x, y ∈ X) hy, xi = α∈bA y(α)x(α). P (e) (∀ x ∈ X) if (∀ α ∈ A) huα, xbi =b 0 then x = 0.

Proof.

(a) ⇒ (b): Let V = span{uα : α ∈ A} and M = V¯ . Then M is a closed subspace. Since ⊥ ⊥ {uα} is maximal, V = {0}, so M = {0}, so M = X. Hilbert Spaces 91

(b) ⇒ (c): Clear if x = 0. Given x =6 0, and given ǫ> 0 (WLOG assume ǫ< kxk), choose

y ∈ V ∋kx − yk < ǫ, say y ∈ span{uα1 ,...,uαk }. Let z = x(α1)uα1 + ··· + x(αk)uαk .

Then z minimizes kx − wk over w ∈ span{uα1 ,...,uαk } so kx − zk≤kx − yk < ǫ. 2 2 2 k b 2 b 2 Thus kxk < kzk + ǫ, so (kxk − ǫ) < kzk and kzk = j=1 |x(αj)| ≤ α∈A |x(α)| . Let ǫ → 0 to get kxk2 ≤ |x(α)|2. The other inequalityP is Bessel’s inequality.P α∈A b b P (c) ⇒ (d): Use polarization. Only countablyb many terms in the sum are nonzero. 2 (d) ⇒ (e): Suppose (∀ α ∈ A) hx, uαi = 0. Then x(α) ≡ 0, so kxk = hx, xi = 0, so x = 0.  (e) ⇒ (a): If {uα} is not maximal, then ∃ x =06 ∋hb x, uαi = 0 for all α ∈ A.

Deﬁnition. An orthonormal set {uα} in a Hilbert space X satisfying the conditions in the previous theorem is called a complete orthonormal set (or a complete orthonormal system) or an orthonormal basis in X. Caution. If X is inﬁnite dimensional, an orthonormal basis is not a basis in the usual deﬁnition of a basis for a vector space (i.e., each x ∈ X has a unique representation as a ﬁnite linear combination of basis elements). Such a basis in this context is called a Hamel basis. Deﬁnition. Let X and Y be inner product spaces. A map T : X → Y which is linear, bijective, and preserves inner products (i.e., (∀ x, y ∈ X) hx, yi = hTx,Tyi — this implies T is an isometry kxk = kT xk) is called a unitary .

Corollary. If X is a Hilbert space and {uα}α∈A is an orthonormal basis of X, then the map 2 F : X → l (A) mapping x 7→ x (where x(α)= hx, uαi) is a unitary isomorphism. 2 Corollary. Every Hilbert spaceb is unitarilyb isomorphic to l (A) for some A.

Norm Convergence of Fourier

Theorem. Let X be a Hilbert space, {uα}α∈A be an orthonormal set in X, and let x ∈ X. 2 2 Let {αj}j≥1 be any enumeration of {α ∈ A : huα, xi=0 6 }. Then kxk = j≥1 |huαj , xi| (i.e. n P Parseval’s Equality holds for this x) iﬀ limn→∞ x − j=1huαj , xiuαj = 0 (i.e. the Fourier ∞ series x(α )u converges to x in norm). P j=1 j αj P Proof. Let bMn = span{uα1 ,...,uαn } and let Pn be the orthogonal projection onto Mn (so ⊥ n 2 I − Pn is the orthogonal projection onto Mn ). Then Pnx = j=1huαj , xiuαj and kPnxk = n 2 2 2 2 2 2 2 j=1 |huαj , xi| . Also kxk = kPnxk + k(I − Pn)xk , so kxk P−kPnxk = kx − Pnxk . Hence 2 2 2 2 2 Pkxk = j≥1 |huαj , xi| iﬀ limn→∞ kPnxk = kxk iﬀ limn→∞ kx − Pnxk = 0, which is the desired conclusion.P (Note: If {α ∈ A : huα, xi=0 6 } is ﬁnite, say {α1,...,αn}, then Parseval 2 2 n holds iﬀ kPnxk = kxk iﬀ x = Pnx, i.e., x = j=1huαj , xiuαj ∈ Mn.) P Corollary. Let {uα}α∈A be an orthonormal set in a Hilbert space X. Then {uα} is an orthonormal basis iﬀ for each x ∈ X and each enumeration {αj}j≥1 of {α ∈ A : huα, xi=0 6 }, n limn→∞ kx − j=1huαj , xiuαj k = 0. P 92 Hilbert Spaces

Cardinality of Orthonormal Bases

Proposition. l2(A) is unitarily isomorphic to l2(B) iﬀ card(A) = card(B). Proposition. Any pair of orthonormal bases in a Hilbert space have the same . Proposition. A Hilbert space X is separable iﬀ it has a countable orthonormal basis. Remark. For a separable Hilbert space X, one can show directly without invoking Zorn’s lemma that X has a countable complete orthonormal set.

Proof. Clear if dim X < ∞. Suppose dim X = ∞. Let z1, z2,... be a countable dense subset. Apply Gram-Schmidt (dropping zero vectors along the way) to get an orthonormal sequence u1,u2,... whose ﬁnite linear combinations include z1,z,..., and thus are dense.  Theorem. (Orthogonal projection in terms of orthonormal bases.) Let X be a Hilbert space, and let M be a closed subspace of X. Let {vβ}β∈B be a complete orthonormal set in ⊥ M, and let {wγ}γ∈C be a complete orthonormal set in M . Then {vβ}∪{wγ} is a complete orthonormal set in X. The orthogonal projection of X onto M is P x = β∈Bhvβ, xivβ, and ⊥ the orthogonal projection of X onto M is Qx = γ∈C hwγ, xiwγ. P P Proof. Follows directly from X = M ⊕ M ⊥ and the projection theorem.  Example. ( in weighted L2 spaces.) Fix a, b ∈ R with −∞ 0 on (a, b) and a w(x)dx < ∞. The function w is called the weight function. For example, take w(x)=(1R− x2)−1/2 on (−1, 1). Deﬁne b 2 2 Lw(a, b)= f : f is measurable on (a, b) and |f(x)| w(x)dx < ∞  Za  b 2 and deﬁne hg, fiw = a g(x)f(x)w(x)dx for f,g ∈ Lw(a, b). Then (after identifying f and g 2 when f = g a.e.), LwR(a, b) is a Hilbert space. 2 Claim. Polynomials are dense in Lw(a, b).

∞ 2 b 2 Proof. First note that if f ∈ L (a, b), then f ∈ Lw(a, b) since |f(x)| w(x)dx ≤ a 1 2 b b R 2 kfk∞ a w(x)dx, and thus kfkw ≤ Mkfk∞, where M = a w(x)dx < ∞. Given R2 1 R  f ∈ Lw(a, b), ∃ g ∈ C[a, b] for which kf − gkw < 2 ǫ (exercise). By the Weierstrass Approx- imation Theorem, polynomials are dense in (C[a, b], k·k∞), so ∃ a polynomial p for which −1 1 1 1 kg −pk∞ < (2M) ǫ. Then kf −pkw ≤kf −gkw +kg −pkw < 2 ǫ+Mkg −pk∞ < 2 ǫ+ 2 ǫ = ǫ.  Applying the Gram-Schmidt process to the linearly independent set {1, x, x2,...} using the 2 inner product of Lw(a, b) produces a sequence of polynomials which are orthonormal in 2 Lw(a, b). 2 Theorem. The orthogonal polynomials in Lw(a, b) are a complete orthonormal set in 2 Lw(a, b). Proof. Finite linear combinations are dense. 