MATH 304 Linear Algebra Lecture 29: Orthogonal Sets. the Gram-Schmidt Process

Home , Orthogonality, Orthogonalization, Orthogonal basis, Orthonormal basis, Standard basis

MATH 304 Linear Algebra Lecture 29: Orthogonal sets. The Gram-Schmidt process. Orthogonal sets Let V be an inner product space with an inner product h·, ·i and the induced norm k · k. Deﬁnition. A nonempty set S ⊂ V of nonzero vectors is called an orthogonal set if all vectors in S are mutually orthogonal. That is, 0 ∈/ S and hx, yi = 0 for any x, y ∈ S, x =6 y. An orthogonal set S ⊂ V is called orthonormal if kxk = 1 for any x ∈ S.

Remark. Vectors v1, v2,..., vk ∈ V form an orthonormal set if and only if 1 if i = j hv , v i = i j 0 if i =6 j Examples. • V = Rn, hx, yi = x · y. The standard basis e1 = (1, 0, 0,..., 0), e2 = (0, 1, 0,..., 0), . . . , en = (0, 0, 0,..., 1). It is an orthonormal set.

• V = R3, hx, yi = x · y. v1 = (3, 5, 4), v2 = (3, −5, 4), v3 = (4, 0, −3). v1 · v2 = 0, v1 · v3 = 0, v2 · v3 = 0, v1 · v1 = 50, v2 · v2 = 50, v3 · v3 = 25.

Thus the set {v1, v2, v3} is orthogonal but not orthonormal. An orthonormal set is formed by v1 v2 1 2 normalized vectors w = kv1k, w = kv2k, v3 3 w = kv3k. π • V = C[−π, π], hf , gi = f (x)g(x) dx. π Z− f1(x) = sin x, f2(x) = sin 2x, ..., fn(x) = sin nx, ...

hfm, fni = sin(mx)sin(nx) dx −π Z π 1 = cos(mx − nx) − cos(mx + nx) dx. −π 2 Z π sin(kx) π cos(kx) dx = =0 if k ∈ Z, k =6 0. −π k x=−π Z π π k =0 =⇒ cos(kx) dx = dx = 2π. −π −π Z Z 1 π hfm, fni = cos(m − n)x − cos(m + n)x dx 2 −π Z π if m = n = 0 if m =6 n

Thus the set {f1, f2, f3,... } is orthogonal but not orthonormal. It is orthonormal with respect to a scaled inner product π 1 hhf , gii = f (x)g(x) dx. π π Z− Orthogonality =⇒ linear independence

Theorem Suppose v1, v2,..., vk are nonzero vectors that form an orthogonal set. Then v1, v2,..., vk are linearly independent.

Proof: Suppose t1v1 + t2v2 + ··· + tk vk = 0 for some t1, t2,..., tk ∈ R. Then for any index 1 ≤ i ≤ k we have

ht1v1 + t2v2 + ··· + tk vk , vi i = h0, vi i = 0.

=⇒ t1hv1, vi i + t2hv2, vi i + ··· + tk hvk , vi i = 0

By orthogonality, ti hvi , vi i = 0 =⇒ ti = 0. Orthonormal bases

Let v1, v2,..., vn be an orthonormal basis for an inner product space V .

Theorem Let x = x1v1 + x2v2 + ··· + xnvn and y = y1v1 + y2v2 + ··· + ynvn, where xi , yj ∈ R. Then

(i) hx, yi = x1y1 + x2y2 + ··· + xnyn, 2 2 2 (ii) kxk = x1 + x2 + ··· + xn . Proof: (ii) followsp from (i) when y = x. n n n n

hx, yi = xi vi , yj vj = xi vi , yj vj * i=1 j=1 + i=1 * j=1 + X n nX Xn X

= xi yj hvi , vj i = xi yi . i=1 j=1 i=1 X X X Let v1, v2,..., vn be a basis for an inner product space V .

Theorem If the basis v1, v2,..., vn is an orthogonal set then for any x ∈ V

hx, v1i hx, v2i hx, vni x = v1 + v2 + ··· + vn. hv1, v1i hv2, v2i hvn, vni

If v1, v2,..., vn is an orthonormal set then

x = hx, v1iv1 + hx, v2iv2 + ··· + hx, vnivn.

Proof: We have that x = x1v1 + ··· + xnvn.

=⇒ hx, vi i = hx1v1 + ··· + xnvn, vi i, 1 ≤ i ≤ n.

=⇒ hx, vi i = x1hv1, vi i + ··· + xnhvn, vi i

=⇒ hx, vi i = xi hvi , vi i. Let V be a vector space with an inner product. Suppose that v1,..., vk ∈ V are nonzero vectors that form an orthogonal set. Given x ∈ V , let

hx, v1i hx, vk i p = v1 + ··· + vk , o = x − p. hv1, v1i hvk , vk i

Let W denote the span of v1,..., vk . Theorem (a) o ⊥ w for all w ∈ W (denoted o ⊥ W ). (b) kok = kx − pk = min kx − wk. w∈W Thus p is the orthogonal projection of the vector x on the subspace W . Also, p is closer to x than any other vector in W , and kok = dist(x, p) is the distance from x to W . Orthogonalization Let V be a vector space with an inner product. Suppose x1, x2,..., xn is a basis for V . Let v1 = x1, hx2, v1i v2 = x2 − v1, hv1, v1i hx3, v1i hx3, v2i v3 = x3 − v1 − v2, hv1, v1i hv2, v2i ...... hxn, v1i hxn, vn−1i vn = xn − v1 −···− vn−1. hv1, v1i hvn−1, vn−1i

Then v1, v2,..., vn is an orthogonal basis for V . The orthogonalization of a basis as described above is called the Gram-Schmidt process.

Normalization

Let V be a vector space with an inner product. Suppose v1, v2,..., vn is an orthogonal basis for V . v1 v2 vn Let w1 = , w2 = ,..., wn = . kv1k kv2k kvnk

Then w1, w2,..., wn is an orthonormal basis for V .

Theorem Any ﬁnite-dimensional vector space with an inner product has an orthonormal basis. Remark. An inﬁnite-dimensional vector space with an inner product may or may not have an orthonormal basis.