Chapter 9 Euclidean Spaces 9.1 Inner Products, Euclidean Spaces The framework of vector spaces allows us deal with ratios of vectors and linear combinations, but there is no way to express the notion of length of a line segment or to talk about orthogonality of vectors. AEuclideanstructurewillallowustodealwithmetric notions such as orthogonality and length (or distance). First, we define a Euclidean structure on a vector space. 507 508 CHAPTER 9. EUCLIDEAN SPACES Definition 9.1. ArealvectorspaceE is a Euclidean space i↵it is equipped with a symmetric bilinear form ': E E R which is also positive definite,which means⇥ that ! '(u, u) > 0, for every u =0. 6 More explicitly, ': E E R satisfies the following axioms: ⇥ ! '(u1 + u2,v)='(u1,v)+'(u2,v), '(u, v1 + v2)='(u, v1)+'(u, v2), '(λu, v)='(u, v), '(u, λv)='(u, v), '(u, v)='(v, u), u =0 impliesthat '(u, u) > 0. 6 The real number '(u, v)isalsocalledtheinner product (or scalar product) of u and v. 9.1. INNER PRODUCTS, EUCLIDEAN SPACES 509 We also define the quadratic form associated with ' as the function Φ: E R+ such that ! Φ(u)='(u, u), for all u E. 2 Since ' is bilinear, we have '(0, 0) = 0, and since it is positive definite, we have the stronger fact that '(u, u)=0 i↵ u =0, that is Φ(u)=0i↵u =0. Given an inner product ': E E R on a vector space E,wealsodenote'(u, v)by⇥ ! u v, or u, v , or (u v), · h i | and Φ(u)by u . k k p 510 CHAPTER 9. EUCLIDEAN SPACES Example 1. The standard example of a Euclidean space is Rn,undertheinnerproduct defined such that · (x ,...,x ) (y ,...,y )=x y + x y + + x y . 1 n · 1 n 1 1 2 2 ··· n n This Euclidean space is denoted by En. Example 2. Let E be a vector space of dimension 2, and let (e1,e2)beabasisofE. If a>0andb2 ac < 0, the bilinear form defined such that − '(x1e1+y1e2,x2e1+y2e2)=ax1x2+b(x1y2+x2y1)+cy1y2 yields a Euclidean structure on E. In this case, 2 2 Φ(xe1 + ye2)=ax +2bxy + cy . 9.1. INNER PRODUCTS, EUCLIDEAN SPACES 511 Example 3. Let [a, b]denotethesetofcontinuousfunc- C tions f :[a, b] R.Itiseasilycheckedthat [a, b]isa vector space of! infinite dimension. C Given any two functions f,g [a, b], let 2C b f,g = f(t)g(t)dt. h i Za We leave as an easy exercise that , is indeed an inner product on [a, b]. h i C When [a, b]=[ ⇡,⇡](or[a, b]=[0, 2⇡], this makes basically no di↵erence),− one should compute sin px, sin qx , sin px, cos qx , h i h i and cos px, cos qx , h i for all natural numbers p, q 1. The outcome of these calculations is what makes Fourier≥ analysis possible! 512 CHAPTER 9. EUCLIDEAN SPACES Example 4. Let E =Mn(R)bethevectorspaceofreal n n matrices. ⇥ If we view a matrix A Mn(R)asa“long”columnvector obtained by concatenating2 together its columns, we can define the inner product of two matrices A, B Mn(R) as 2 n A, B = a b , h i ij ij i,j=1 X which can be conveniently written as A, B =tr(A>B)=tr(B>A). h i Since this can be viewed as the Euclidean product on n2 R ,itisaninnerproductonMn(R). The corresponding norm A = tr(A A) k kF > q is the Frobenius norm (see Section 6.2). 9.1. INNER PRODUCTS, EUCLIDEAN SPACES 513 Let us observe that ' can be recovered from Φ. Indeed, by bilinearity and symmetry, we have Φ(u + v)='(u + v, u + v) = '(u, u + v)+'(v, u + v) = '(u, u)+2'(u, v)+'(v, v) =Φ(u)+2'(u, v)+Φ(v). Thus, we have 1 '(u, v)= [Φ(u + v) Φ(u) Φ(v)]. 2 − − We also say that ' is the polar form of Φ. If E is finite-dimensional and if ': E E R is a ⇥ ! bilinear form on E,givenanybasis(e1,...,en)ofE,we n n can write x = i=1 xiei and y = j=1 yjej,andwehave P n P '(x, y)= xiyj'(ei,ej). i,j=1 X 514 CHAPTER 9. EUCLIDEAN SPACES If we let G be the matrix G =('(ei,ej)), and if x and y are the column vectors associated with (x1,...,xn)and (y1,...,yn), then we can write '(x, y)=x>Gy = y>G>x. Note that we are committing an abuse of notation, since n x = i=1 xiei is a vector in E,butthecolumnvector n associated with (x1,...,xn)belongstoR . P To avoid this minor abuse, we could denote the column vector associated with (x1,...,xn)byx (and similarly y for the column vector associated with (y1,...,yn)), in wich case the “correct” expression for '(x, y)is '(x, y)=x>Gy. However, in view of the isomorphism between E and Rn, to keep notation as simple as possible, we will use x and y instead of x and y. 9.1. INNER PRODUCTS, EUCLIDEAN SPACES 515 The matrix G associated with an inner product is called the Gram matrix of the inner product with respect to the basis (e1,...,en). Proposition 9.1. Let E be a finite-dimensional vec- tor space, and let (e1,...,en) be a basis of E. 1. For any inner product , on E, if G =(ei,ej ) is the Gram matrix ofh thei inner product h , i h i w.r.t. the basis (e1,...,en), then G is symmetric positive definite. 2. For any change of basis matrix P , the Gram ma- trix of , with respect to the new basis is P >GP . h i 3. If A is any n n symmetric positive definite ma- trix, then ⇥ x, y = x>Ay h i is an inner product on E. One of the very important properties of an inner product ' is that the map u Φ(u)isanorm. 7! p 516 CHAPTER 9. EUCLIDEAN SPACES Proposition 9.2. Let E be a Euclidean space with inner product ' and quadratic form Φ. For all u, v E, we have the Cauchy-Schwarz inequality: 2 '(u, v)2 Φ(u)Φ(v), the equality holding i↵ u and v are linearly dependent. We also have the Minkovski inequality: Φ(u + v) Φ(u)+ Φ(v), the equalityp holding i↵ u andp v are linearlyp dependent, where in addition if u =0and v =0, then u = λv for some λ>0. 6 6 9.1. INNER PRODUCTS, EUCLIDEAN SPACES 517 Sketch of proof .DefinethefunctionT : R R,such that ! T (λ)=Φ(u + λv), for all λ R.Usingbilinearityandsymmetry,wecan show that2 Φ(u + λv)=Φ(u)+2'(u, v)+λ2Φ(v). Since ' is positive definite, we have T (λ) 0forall ≥ λ R. 2 If Φ(v)=0,thenv =0,andwealsohave'(u, v)=0. In this case, the Cauchy-Schwarz inequality is trivial, 518 CHAPTER 9. EUCLIDEAN SPACES If Φ(v) > 0, then λ2Φ(v)+2'(u, v)+Φ(u)=0 can’t have distinct roots, which means that its discrimi- nant ∆=4('(u, v)2 Φ(u)Φ(v)) − is zero or negative, which is precisely the Cauchy-Schwarz inequality. The Minkovski inequality can then be shown. 9.1. INNER PRODUCTS, EUCLIDEAN SPACES 519 The Minkovski inequality Φ(u + v) Φ(u)+ Φ(v) shows that thep map u p Φ(u)satisfiesthep triangle inequality,condition(N3)ofdefinition6.1,andsince7! ' p is bilinear and positive definite, it also satisfies conditions (N1) and (N2) of definition 6.1, and thus, it is a norm on E. The norm induced by ' is called the Euclidean norm induced by '. Note that the Cauchy-Schwarz inequality can be written as u v u v , | · |k kk k and the Minkovski inequality as u + v u + v . k kk k k k 520 CHAPTER 9. EUCLIDEAN SPACES Remark: One might wonder if every norm on a vector space is induced by some Euclidean inner product. In general, this is false, but remarkably, there is a simple necessary and sufficient condition, which is that the norm must satisfy the parallelogram law: u + v 2 + u v 2 =2( u 2 + v 2). k k k − k k k k k If , is an inner product, then we have h i u + v 2 = u 2 + v 2 +2 u, v k k k k k k h i u v 2 = u 2 + v 2 2 u, v , k − k k k k k − h i and by adding and subtracting these identities, we get the parallelogram law, and the equation 1 u, v = ( u + v 2 u v 2), h i 4 k k k − k which allows us to recover , from the norm. h i 9.1. INNER PRODUCTS, EUCLIDEAN SPACES 521 Conversely, if is a norm satisfying the parallelogram law, and if it comeskk from an inner product, then this inner product must be given by 1 u, v = ( u + v 2 u v 2). h i 4 k k k − k Proving that the above form is indeed symmetric and bilinear is quite tricky. We now define orthogonality. 522 CHAPTER 9. EUCLIDEAN SPACES 9.2 Orthogonality, Duality, Adjoint Maps Definition 9.2. Given a Euclidean space E,anytwo vectors u, v E are orthogonal, or perpendicular i↵ 2 u v =0.Givenafamily(ui)i I of vectors in E,wesay · 2 that (ui)i I is orthogonal i↵ ui uj =0foralli, j I, 2 · 2 where i = j.Wesaythatthefamily(ui)i I is orthonor- mal i↵ 6u u =0foralli, j I,where2 i = j,and i · j 2 6 ui = ui ui =1,foralli I.ForanysubsetF of E, thek k set · 2 F ? = v E u v =0, for all u F , { 2 | · 2 } of all vectors orthogonal to all vectors in F ,iscalledthe orthogonal complement of F .