sign in sign up
<<
Home , Function space, Scalar (mathematics), Scalar multiplication, Scaling (geometry), Weight function

Jim Lambers MAT 606 Spring Semester 2015-16 Lecture 18 Notes

These notes correspond to Section 6.3 in the text.

Vectors in Spaces

We begin with some necessary terminology. A vector V , also known as a linear , is a of objects, called vectors, together with two operations:

• of two vectors in V , which must be commutative, associative, and have an identity element, which is the zero vector 0. Each vector v must have an −v which, when added to v, yields the zero vector.

• of a vector in V by a , which is typically a real or complex . The term “scalar” is used in this context, rather than “number”, because the multiplication process is “” a given vector by a factor indicated by a given number. must satisfy distributive laws, and have an identity element, 1, such that 1v = v for any vector v ∈ V .

Both operations must be closed, which that the result of either must be a vector in V . That is, if u and v are two vectors in V , then u + v must also be in V , and αv must be in V for any scalar α.

n Example 1 The set of all points in n-dimensional space, R , is a vector space. Addition is defined as follows:       u1 v1 u1 + v1  u2   v2   u2 + v2  u + v =   +   =   = u + v.  .   .   .   .   .   .  un vn un + vn Scalar multiplication is defined by   αv1  αv2  αv =   .  .   .  αvn Similarly, the set of all n-dimensional points whose coordinates are complex , denoted by n C , is also a vector space. 2

In these next few examples, we introduce some vector spaces whose vectors are functions, which are also known as function spaces.

Example 2 The set of all of degree at most n, denoted by Pn, is a vector space, in which addition and scalar multiplication are defined as follows. Given f(x), g(x) ∈ Pn,

(f + g)(x) = f(x) + g(x), (αf)(x) = αf(x).

1 These operations are closed, because adding two polynomials of degree at most n will not yield a sum whose degree is greater than n, and multiplying any by a nonzero scalar will not change its degree. 2

Example 3 The set of all functions with power series of the form

∞ X n f(x) = anx , n=0 that are convergent on the (−1, 1) is a vector space, in which addition and multiplication are defined as in the previous example. These operations are closed because the sum of two convergent series is also convergent, as is a scalar multiple of a convergent series. 2

Example 4 The set of all continuous functions on the interval [a, b], denoted by C[a, b], is a vector space in which addition and scalar multiplication are defined as in the previous two examples. These operations are closed because the sum of two continuous functions, and a scalar multiple of a , is also continuous. 2

A vector space V is most effectively described in terms of a set of specific vectors {v1, v2,...} that, in conjunction with the operations of addition and scalar multiplication, can be used to obtain every vector in the space. That is, for every vector v ∈ V , there must exist scalars c1, c2,..., such that v = c1v1 + c2v2 + ··· .

We say that v is a of v1, v2,..., and the scalars c1, c2,... are the coefficients of the linear combination. Ideally, it should be possible to express any vector v ∈ V as a unique linear combination of the vectors v1, v2,... that are to be used to describe all vectors in V . With this criteria in mind, we introduce the following two essential concepts from linear :

• A set of vectors {v1, v2,..., vn} is linearly independent if the vector equation

c1v1 + c2v2 + ··· + anvn = 0

is satisfied if and only if c1 = c2 = ··· = 0. In other words, this set of vectors is linearly independent if it is not possible to express any vector in the set as a linear combination of other vectors in the set. This definition can be generalized in a natural way to an infinite set of vectors. If a set of vectors is not linearly independent, then we say that it is linearly dependent.

• A set of vectors {v1, v2,..., vn} spans a vector space V if, for any vector v ∈ V , there exist scalars c1, c2, . . . , an such that

v = c1v1 + c2v2 + ··· + anvn.

That is, any vector in V can be expressed as a linear combination of vectors in the set. We define span{v1, v2,..., vn} to be the set of all linear combinations of v1, v2,..., vn. As with , the notion of span generalizes naturally to an infinite set of vectors.

2 We then say that a set of vectors {v1, v2,...} (which may be finite or infinite) is a for a vector space V if it is linearly independent, and if it spans V . This definition ensures that any vector in V is a unique linear combination of the vectors in the basis. If a basis for V is finite, then we say that V is finite-dimensional and define the of V to be the number of elements in a basis; all bases of a finite-dimensional vector space must have the same number of elements. If V does not have a finite basis, then we say that V is infinite-dimensional.

Example 5 The P3, consisting of polynomials of degree at most 3, has a basis 2 3 {1, x, x , x }. It is clear that any polynomial in P3 can be expressed as a linear combination of these basis functions, as the coefficients of any such polynomial are also the coefficients in the linear combination of these basis functions. To confirm linear independence, suppose that there exists constants c0, c1, c2, and c3 such that 2 3 c0(1) + c1x + c2x + c3x = 0 for all x ∈ R. Then certainly this must be the case at x = 0, which requires that c1 = 0. Substituting 3 other values of x into the above equation yields a system of 3 linear equations in the remaining 3 unknows c1, c2 and c3. It can be shown that the only solution of such a system of equations is 2 3 the trivial solution c1 = c2 = c3 = 0. Therefore the set {1, x, x , x } is linearly independent. An alternative basis consists of the first 4 Chebyshev polynomials {1, x, 2x2 − 1, 4x3 − 3x}. It can be confirmed using a similar approach that these polynomials are linearly independent. 2

Example 6 The function space consisting of all power series that are convergent on the interval (−1, 1) has as a basis the infinite set {1, x, x2, x3,...}. Using an inductive argument, it can be shown that this set is linearly independent 2

Inner n Recall that the of two vectors u and v in R is

u · v = u1v1 + u2v2 + ··· + unvn = kukkvk cos θ, where q 2 2 2 kuk = u1 + u2 + ··· + un is the or length of u, and θ is the between u and v, with 0 ≤ θ ≤ π radians. The dot product has the following properties: 1. u · u = kuk2 2. u · (v + w) = u · v + u · w 3. u · v = v · u 4. u · (cv) = c(u · v) When u and v are perpendicular, then cos θ = 0. It follows that u · v = 0, and we say that u and v are orthogonal. We would like to generalize the concept of a dot product to vectors in function spaces, and we also need to ensure that complex numbers are properly taken into account. To that end, we define the inner product of two functions f(x) and g(x) to be Z b hf, gi = f(x)g(x)w(x) dx, a

3 where w(x) is a weight function and, for any z = x + iy, z = x − iy is the complex conjugate of z. The interval of integration [a, b] depends on the function space under consideration. Using this definition, it can be verified that the inner product has the following properties: 1. hf, g + hi = hf, gi + hf, hi 2. hf, gi = hg, fi 3. hf, cgi = chf, gi for any complex number c Note that the second property is slightly different from the corresponding property for vectors in n R , as it requires the complex conjugate. Combining the second and third property yields the result hcf, gi = chf, gi.

Inner Product Spaces and Hilbert Spaces n Just as we use kvk to the magnitude of a vector v ∈ R , we need a notion of magnitude for a function f(x) in a function space. To that end, we say that a function k · k : V → R is a on a vector space V if it satisfies the following conditions: 1. kvk ≥ 0 for any vector v ∈ V , and kvk = 0 if and only if v = 0. 2. kαvk = |α|kvk for any complex scalar α. 3. ku + vk ≤ kuk + kvk for any two vectors u, v ∈ V . This is known as the Triangle inequality. Any function that satisfies these conditions is useful for measuring the magnitude of a vector. Given an inner product h·, ·i, we define kvk = hv, vi1/2 to be the norm induced by this inner product. Using the properties of the inner product, it can be shown that this norm satisfies the above conditions, except that for a function space, kfk may equal zero even if f is nonzero at isolated points, or more generally, if f is nonzero on a set of measure zero. We can now define a specific type of function space that will be of use to us. An is a vector space, together with an inner product and induced norm. A H is a function space, together with an inner product and induced norm, that is also complete. This means that there exists a basis ϕ1, ϕ2,... such that for any f ∈ H, there exists scalars c1, c2,... such that ∞ X f = anϕn. n=1

Example 7 As before, let P3 be the space of polynomials of degree at most 3. If we use the inner product Z 1 hf, gi = f(s)g(s) ds, −1 with weight function w(x) = 1, then P3 is a Hilbert space. Let L0(x) = 1 and L1(x) = x. Then Z 1 Z 1 2 hL0,L0i = 1 ds = 2 1 ds = 2, −1 0 Z 1 Z 1 2 2 2 hL1,L1i = x ds = 2 x dx = , −1 0 3 Z 1 hL0,L1i = 1(x) ds = 0. −1

4 Here, we have used the fact that if f(x) is an even function, meaning that f(−x) = f(x), then Z a Z a f(x) dx = 2 f(x) dx, −a 0 whereas if f(x) is an odd function, meaning that f(−x) = −f(x), then Z a f(x) dx = 0. −a 2

Example 8 We denote by E(a, b) the inner product space consisting of all piecewise continuous functions on [a, b]. Its inner product is the standard inner product on (a, b),

Z b hf, gi = f(x)g(x) dx. a

More generally, we denote by Er(a, b) the inner product space of all piecewise continuous functions on [a, b], with inner product Z b hf, gir = f(x)g(x)r(x) dx, a where r is a weight function. It must be continuous and positive on [a, b]. It will be seen later that Er(a, b), for such a weight function r, is actually a Hilbert space, as it is complete. 2

Cauchy-Schwarz Inequality The Cauchy-Schwarz inequality, also known as the Schwarz inequality, states that if a norm on a vector space V is defined by kfk = hf, fi1/2 for any f ∈ V , where hf, gi is an inner product as defined previously, then |hf, gi| ≤ kfkkgk. We will prove this inequality in the case where V is a vector space defined over the real numbers; the proof can be generalized to a complex vector space. For f, g ∈ V and c ∈ R, with g 6= 0, we have hf − cg, f − cgi ≥ 0. It follows from the properties of the inner product that

0 ≤ hf − cg, f − cgi ≤ hf, fi − hf, cgi − hcg, fi + hcg, cgi ≤ kfk2 − 2chf, gi + c2kgk2.

We now try to find the value of c that minimizes this expression. Differentiating with respect to c and equating to zero yields the equation

−2hf, gi + 2ckgk2 = 0,

5 and therefore the minimum occurs when c = hf, gi/kgk2. It follows that

0 ≤ kfk2 − 2chf, gi + c2kgk2 hf, gi hf, gi2 ≤ kfk2 − 2 hf, gi + kgk2 kgk2 kgk4 hf, gi2 hf, gi2 ≤ kfk2 − 2 + kgk2 kgk2 hf, gi2 ≤ kfk2 − . kgk2

It follows that hf, gi2 ≤ kfk2kgk2. Taking the of both sides yields the Cauchy-Schwarz inequality.

Orthogonal Expansions n Just as two vectors u, v ∈ R are orthogonal if u · v = 0, we say that two vectors u and v in an inner product space are orthogonal if hu, vi = 0. An of an Hilbert space is particularly useful because the coefficients of a function with respect to such a basis are easily computed. Specifically, suppose a Hilbert space H has a basis {ϕ1, ϕ2,...} such that hϕi, ϕji = 0 for i 6= j. Then, let f ∈ H have the expansion

∞ X f = anϕn. n=1

If we take the inner product of both sides with ϕk for some positive integer k, we obtain

* ∞ + ∞ X X hϕk, fi = ϕk, anϕn = anhϕk, ϕni = akhϕk, ϕki, n=1 n=1 which yields the coefficients hϕn, fi an = , n = 1, 2,.... hϕn, ϕni This is significant because it shows that the coefficients can be computed independently of one another. n Recall that a vector u ∈ R is a unit vector if kuk = 1, and that a unit vector can be obtained from a nonzero vector v by normalizing v, which means dividing by its magnitude: u = v/kvk is a unit vector. Similarly, given a function f in a Hilbert space H, f is said to be normalized if kfk = 1. We then say that a set of function {ϕ1, ϕ2,...} is orthonormal if

hϕi, ϕji = δij, where  0 i 6= j δ = ij 1 i = j is called the Kronecker delta.

6 ∞ Example 9 Consider the infinite set of functions {sin nx}n=1, with the inner product Z π hf, gi = f(s)g(s) ds. 0 Then we have, for positive integers m and n, with m 6= n, Z π hsin mx, sin nxi = sin mx sin nx dx 0 1 Z π = cos[(m − n)x] − cos[(m + n)x] dx 2 0   π 1 1 = sin[(m − n)x] − sin[(m + n)x] m − n m + n 0 = 0, k sin nxk2 = hsin nx, sin nxi Z π = sin2 nx dx 0 Z π 1 − cos 2nx = dx 0 2 π x − 1 sin 2nx = 2n 2 0 π = . 2 Therefore, the functions in this set are orthogonal, but not orthonormal. Since the norm of each is pπ/2, it follows that the set ( )∞ r 2 sin nx π n=1 ∞ is an orthonormal set. Then, to compute the coefficients {an}n=1 in the expansion r ∞ 2 X f(x) = a sin nx, π n n=1 we need only compute the inner products * + r 2 r 2 Z π an = sin nx, f = f(x) sin nx dx, n = 0, 1, 2,.... π π 0 This representation of f(x) is called a Fourier sine series. 2

An expansion of a function f in an orthonormal basis {ϕn}, ∞ X f ≈ fˆnϕn, fˆn = hϕn, fi, n = 1, 2,..., n=1 is called a (generalized) Fourier expansion of f. The coefficients {fˆn} are called the (generalized) Fourier coefficients of f. A truncated Fourier expansion

N X ˆ fN = fnϕn n=1

7 is called the orthogonal projection of f onto the subspace

VN = span{ϕ1, ϕ2, . . . , ϕN }.

This orthogonal projection satisfies v u N u 2 X ˆ 2 kf − fN k = tkfk − |fn| ≤ kf − vk, v ∈ VN . n=1

Taking the limit as N → ∞, and using the fact that kf − fN k ≥ 0, we obtain Bessel’s inequality

∞ X 2 2 |fˆn| ≤ kfk . n=1 An immediate consequence of this inequality is the Riemann-Lebesgue Lemma

lim fˆn = lim hϕn, fi = 0. n→∞ n→∞ If the underlying function space is a Hilbert space, meaning that it is complete, then

lim kf − fN k = 0, N→∞ and we say that the Fourier expansion converges in the to f. It then follows from the properties of the norm and inner product that

∞ X 2 2 |fˆn| = kfk . n=1 This is known as Parseval’s identity. It is important to note that even if the Fourier expansion of f converges to f in the mean, this does not necessarily mean that the expansion agrees with f at every single point in the domain on which the inner product is defined. This is because, as mentioned earlier, that hf, fi = 0 is possible even if f is nonzero at certain isolated points in the interval of integration. This will be illustrated in the following example.

Example 10 Consider the function f(x) = x on (−π, π). We wish to obtain an expansion of this function in terms of the functions cos nx, n = 0, 1, 2,... and sin nx, n = 1, 2,..., which are known to be orthogonal with respect to the standard inner product on (−π, π), Z π hf, gi = f(s)g(s) ds. −π Therefore, our expansion will have the form

∞ X f(x) = a0 + (an cos nx + bn sin nx). n=1

8 Since f(x) is an odd function, an = 0 for n = 0, 1, 2,..., so we need only compute the coefficients ∞ {bn}n=1, which are given by hsin nx, fi b = n hsin nx, sin nxi 1 Z π = f(x) sin nx dx π −π 1 Z π = x sin nx dx π −π  π Z π  1 1 1 = − x cos nx + cos nx dx π n −π n −π  π  1 1 1 = − x cos nx + 2 sin nx π n n −π 2 = − cos nπ n 2(−1)n+1 = . n Therefore, the Fourier sine series for f(x) = x on (−π, π) is

∞ X (−1)n+1 x = 2 sin nx. n n=1 Figure 1 shows how this series converges to f(x). Note that because the 2π-periodic extension of f(x) has a discontinuity at x = kπ, where k is any odd integer, at these points the series converges to the average of the values of f(x) at these jumps, which is 0. The resulting oscillations near the discontinuity are an example of Gibbs’ phenomenon. 2

Expansions and Inner Products Suppose that two functions f and g in a Hilbert space are expanded in the same orthonormal basis:

∞ ∞ X X f = fˆnϕn, g = gˆmϕm. n=1 m=1 Then, using the properties of the inner product, we have

* ∞ ∞ + X X hf, gi = fˆnϕn, gˆmϕm n=1 m=1 ∞ ∞ X X = fˆngˆmhϕn, ϕmi n=1 m=1 ∞ ∞ X X = fˆngˆmδnm n=1 m=1 ∞ X = fˆngˆn. n=1

9 Figure 1: Fourier sine series expansion of f(x) = x (red curve), compared to f(x) itself (blue curve)

In other words, the inner product reduces to a dot product:

hf, gi = f H g, where f and g are “column vectors” (which may be infinitely long) consisting of the coefficients H {fˆn} and {gˆm}, respectively. For any vector v, we denote by v the Hermitian of v, which is the transpose and complex conjugate of v. In some texts, vH is written as vH or v†. Note that in the case of f = g, we obtain

∞ 2 X 2 kfk = hf, fi = |fˆn| , n=1 which is again Parseval’s identity.

Example 11 The functions

r 1 r 2 ϕ (x) = , ϕ (x) = cos nx, n = 1, 2,... 0 π n π form an orthonormal set with respect to the inner product Z π hf, gi = f(s)g(s) ds. 0 Now, consider the functions

3 2 2 ψ1(x) = cos x + sin x + cos x + 1, ψ2(x) = cos x − cos x.

10 It can be verified directly by computing hϕj, ψki for j = 0, 1, 2, 3 and k = 1, 2, or by rewriting ψ1 and ψ2 using trigonometric identities, that these functions have the following expansions in the ∞ basis {ϕn}n=0: √ √ √ √ 3 π 7 π π π ψ1(x) = ϕ0(x) + √ ϕ1(x) − √ ϕ2(x) + √ ϕ3(x), 2 4 2 2 2 4 2 √ √ π rπ π ψ2(x) = ϕ0(x) − ϕ1(x) + √ ϕ2(x). 2 2 2 2

Because the basis is orthonormal, we can compute hψi, ψji, for i, j = 1, 2, by computing dot products of the appropriate sets of coefficients: √ √ √ √ 3 π 2 7 π 2  π 2  π 2 hψ1, ψ1i = + √ + − √ + √ 2 4 2 2 2 4 2 63π = , 16 √ √ √ √ √ √ 3 π π 7 π rπ π π π hψ1, ψ2i = − √ − √ √ + √ (0) 2 2 4 2 2 2 2 2 2 4 2 π = − , 4 √ √  π 2 rπ 2  π 2 hψ1, ψ2i = + + √ 2 2 2 2 7π = . 8 2

11