Chapter 6 Euclidean Spaces 6.1 Inner Products, Euclidean Spaces The framework of vector spaces allows us deal with ratios of vectors and linear combinations, but there is no way to express the notion of length of a line segment or to talk about orthogonality of vectors. AEuclideanstructurewillallowustodealwithmetric notions such as orthogonality and length (or distance). First, we define a Euclidean structure on a vector space. 419 420 CHAPTER 6. EUCLIDEAN SPACES Definition 6.1. ArealvectorspaceE is a Euclidean space i↵it is equipped with a symmetric bilinear form ': E E R which is also positive definite,which means⇥ that ! '(u, u) > 0, for every u =0. 6 More explicitly, ': E E R satisfies the following axioms: ⇥ ! '(u1 + u2,v)='(u1,v)+'(u2,v), '(u, v1 + v2)='(u, v1)+'(u, v2), '(λu, v)='(u, v), '(u, λv)='(u, v), '(u, v)='(v, u), u =0 impliesthat '(u, u) > 0. 6 The real number '(u, v)isalsocalledtheinner product (or scalar product) of u and v. 6.1. INNER PRODUCTS, EUCLIDEAN SPACES 421 We also define the quadratic form associated with ' as the function Φ: E R+ such that ! Φ(u)='(u, u), for all u E. 2 Since ' is bilinear, we have '(0, 0) = 0, and since it is positive definite, we have the stronger fact that '(u, u)=0 i↵ u =0, that is Φ(u)=0i↵u =0. Given an inner product ': E E R on a vector space E,wealsodenote'(u, v)by⇥ ! u v, or u, v , or (u v), · h i | and Φ(u)by u . k k p 422 CHAPTER 6. EUCLIDEAN SPACES Example 1. The standard example of a Euclidean space is Rn,undertheinnerproduct defined such that · (x ,...,x ) (y ,...,y )=x y + x y + + x y . 1 n · 1 n 1 1 2 2 ··· n n This Euclidean space is denoted by En. Example 2. Let E be a vector space of dimension 2, and let (e1,e2)beabasisofE. If a>0andb2 ac < 0, the bilinear form defined such that − '(x1e1+y1e2,x2e1+y2e2)=ax1x2+b(x1y2+x2y1)+cy1y2 yields a Euclidean structure on E. In this case, 2 2 Φ(xe1 + ye2)=ax +2bxy + cy . 6.1. INNER PRODUCTS, EUCLIDEAN SPACES 423 Example 3. Let [a, b]denotethesetofcontinuousfunc- C tions f :[a, b] R.Itiseasilycheckedthat [a, b]isa vector space of! infinite dimension. C Given any two functions f,g [a, b], let 2C b f,g = f(t)g(t)dt. h i Za We leave as an easy exercise that , is indeed an inner product on [a, b]. h i C When [a, b]=[ ⇡,⇡](or[a, b]=[0, 2⇡], this makes basically no di↵erence),− one should compute sin px, sin qx , sin px, cos qx , h i h i and cos px, cos qx , h i for all natural numbers p, q 1. The outcome of these calculations is what makes Fourier≥ analysis possible! 424 CHAPTER 6. EUCLIDEAN SPACES Example 4. Let E =Mn(R)bethevectorspaceofreal n n matrices. ⇥ If we view a matrix A Mn(R)asa“long”columnvector obtained by concatenating2 together its columns, we can define the inner product of two matrices A, B Mn(R) as 2 n A, B = a b , h i ij ij i,j=1 X which can be conveniently written as A, B =tr(A>B)=tr(B>A). h i Since this can be viewed as the Euclidean product on n2 R ,itisaninnerproductonMn(R). The corresponding norm A = tr(A A) k kF > q is the Frobenius norm (see Section 4.2). 6.1. INNER PRODUCTS, EUCLIDEAN SPACES 425 Let us observe that ' can be recovered from Φ. Indeed, by bilinearity and symmetry, we have Φ(u + v)='(u + v, u + v) = '(u, u + v)+'(v, u + v) = '(u, u)+2'(u, v)+'(v, v) =Φ(u)+2'(u, v)+Φ(v). Thus, we have 1 '(u, v)= [Φ(u + v) Φ(u) Φ(v)]. 2 − − We also say that ' is the polar form of Φ. One of the very important properties of an inner product ' is that the map u Φ(u)isanorm. 7! p 426 CHAPTER 6. EUCLIDEAN SPACES Proposition 6.1. Let E be a Euclidean space with inner product ' and quadratic form Φ. For all u, v E, we have the Cauchy-Schwarz inequality: 2 '(u, v)2 Φ(u)Φ(v), the equality holding i↵ u and v are linearly dependent. We also have the Minkovski inequality: Φ(u + v) Φ(u)+ Φ(v), the equalityp holding i↵ u andp v are linearlyp dependent, where in addition if u =0and v =0, then u = λv for some λ>0. 6 6 6.1. INNER PRODUCTS, EUCLIDEAN SPACES 427 Sketch of proof .DefinethefunctionT : R R,such that ! T (λ)=Φ(u + λv), for all λ R.Usingbilinearityandsymmetry,wecan show that2 Φ(u + λv)=Φ(u)+2'(u, v)+λ2Φ(v). Since ' is positive definite, we have T (λ) 0forall ≥ λ R. 2 If Φ(v)=0,thenv =0,andwealsohave'(u, v)=0. In this case, the Cauchy-Schwarz inequality is trivial, 428 CHAPTER 6. EUCLIDEAN SPACES If Φ(v) > 0, then λ2Φ(v)+2'(u, v)+Φ(u)=0 can’t have distinct roots, which means that its discrimi- nant ∆=4('(u, v)2 Φ(u)Φ(v)) − is zero or negative, which is precisely the Cauchy-Schwarz inequality. The Minkovski inequality can then be shown. 6.1. INNER PRODUCTS, EUCLIDEAN SPACES 429 The Minkovski inequality Φ(u + v) Φ(u)+ Φ(v) shows that thep map u p Φ(u)satisfiesthep triangle inequality,condition(N3)ofdefinition4.1,andsince7! ' p is bilinear and positive definite, it also satisfies conditions (N1) and (N2) of definition 4.1, and thus, it is a norm on E. The norm induced by ' is called the Euclidean norm induced by '. Note that the Cauchy-Schwarz inequality can be written as u v u v , | · |k kk k and the Minkovski inequality as u + v u + v . k kk k k k We now define orthogonality. 430 CHAPTER 6. EUCLIDEAN SPACES 6.2 Orthogonality, Duality, Adjoint Maps Definition 6.2. Given a Euclidean space E,anytwo vectors u, v E are orthogonal, or perpendicular i↵ 2 u v =0.Givenafamily(ui)i I of vectors in E,wesay · 2 that (ui)i I is orthogonal i↵ ui uj =0foralli, j I, 2 · 2 where i = j.Wesaythatthefamily(ui)i I is orthonor- mal i↵ 6u u =0foralli, j I,where2 i = j,and i · j 2 6 ui = ui ui =1,foralli I.ForanysubsetF of E, thek k set · 2 F ? = v E u v =0, for all u F , { 2 | · 2 } of all vectors orthogonal to all vectors in F ,iscalledthe orthogonal complement of F . Since inner products are positive definite, observe that for any vector u E,wehave 2 u v =0 forallv E i↵ u =0. · 2 It is immediately verified that the orthogonal complement F ? of F is a subspace of E. 6.2. ORTHOGONALITY, DUALITY, ADJOINT MAPS 431 Example 5. Going back to example 3, and to the inner product ⇡ f,g = f(t)g(t)dt h i ⇡ Z− on the vector space [ ⇡,⇡], it is easily checked that C − ⇡ if p = q, p, q 1, sin px, sin qx = ≥ h i 0ifp = q, p, q 1 ⇢ 6 ≥ ⇡ if p = q, p, q 1, cos px, cos qx = ≥ h i 0ifp = q, p, q 0 ⇢ 6 ≥ and sin px, cos qx =0, h i for all p 1andq 0, and of course, ≥⇡ ≥ 1, 1 = ⇡ dx =2⇡. h i − R As a consequence, the family (sin px)p 1 (cos qx)q 0 is orthogonal. ≥ [ ≥ It is not orthonormal, but becomes so if we divide every trigonometric function by p⇡,and1byp2⇡. 432 CHAPTER 6. EUCLIDEAN SPACES Proposition 6.2. Given a Euclidean space E, for any family (ui)i I of nonnull vectors in E, if (ui)i I is or- thogonal, then2 it is linearly independent. 2 Proposition 6.3. Given a Euclidean space E, any two vectors u, v E are orthogonal i↵ 2 u + v 2 = u 2 + v 2 . k k k k k k One of the most useful features of orthonormal bases is that they a↵ord a very simple method for computing the coordinates of a vector over any basis vector. 6.2. ORTHOGONALITY, DUALITY, ADJOINT MAPS 433 Indeed, assume that (e1,...,em)isanorthonormalbasis. For any vector x = x e + + x e , 1 1 ··· m m if we compute the inner product x e ,weget · i x e = x e e + + x e e + + x e e = x , · i 1 1 · i ··· i i · i ··· m m · i i since 1ifi = j, e e = i · j 0ifi = j, ⇢ 6 is the property characterizing an orthonormal family. Thus, x = x e , i · i which means that xiei =(x ei)ei is the orthogonal projec- tion of x onto the subspace· generated by the basis vector ei. If the basis is orthogonal but not necessarily orthonormal, then x ei x ei xi = · = · 2. ei ei e · k ik 434 CHAPTER 6. EUCLIDEAN SPACES All this is true even for an infinite orthonormal (or or- thogonal) basis (ei)i I. 2 However, remember that every vector x is expressed as alinearcombination x = xiei i I X2 where the family of scalars (xi)i I has finite support, 2 which means that xi =0foralli I J,whereJ is a finite set. 2 − 6.2. ORTHOGONALITY, DUALITY, ADJOINT MAPS 435 Thus, even though the family (sin px)p 1 (cos qx)q 0 is orthogonal (it is not orthonormal, but≥ becomes[ one≥ if we divide every trigonometric function by p⇡,and1by p2⇡;wewon’tbecauseitlooksmessy!),thefactthata function f 0[ ⇡,⇡]canbewrittenasaFourierseries as 2C − 1 f(x)=a0 + (ak cos kx + bk sin kx) Xk=1 does not mean that (sin px)p 1 (cos qx)q 0 is a basis of this vector space of functions,≥ [ because in≥ general, the families (ak)and(bk) do not have finite support! In order for this infinite linear combination to make sense, it is necessary to prove that the partial sums n a0 + (ak cos kx + bk sin kx) Xk=1 of the series converge to a limit when n goes to infinity.