INNER PRODUCT SPACES • Motivation and Definitions • Inner Product Spaces as Normed Spaces EL3370 Inner Product Spaces - 1 MOTIVATION AND DEFINITIONS n Consider the space ! . It has: 1. Vector space (algebraic) structure: n Given !x, y ∈ , their sum !x + y and scalar multiplication !α x ( α ∈) are defined 2. Inner product structure: n n (x, y)= x y x = (x ,…,x ), y = (x ,…,x )∈ (!x : complex conjugate of !x ∈) ∑ i i ! 1 n 1 n ! i=1 Many physical properties (e.g. work) can be defined in terms of inner products Also, !( ⋅ ,⋅) can define: distances (metrics), length (norms), angles, limits (topologies), ... Goal: Extend inner products to general (possibly infinite dimensional) vector spaces EL3370 Inner Product Spaces - 2 MOTIVATION AND DEFINITIONS (CONT.) Definition Let l2 denote the vector space over of all complex sequences x = {xn } which are square ! ! ∞ 2 summable, i.e., that satisFy x < ∞, with componentwise addition and scalar !∑n=1 n multiplication: x + y := {x + y }, x = {x }, y = { y }∈l n n n n 2 α x := {α x }, α ∈! ! n ∞ and inner product: (x, y):= x y ! ∑n=1 n n Observation Need to verify that these operations (sum, scalar multiplication, inner product) are valid! (we will do it later, using Cauchy-Schwarz inequality) EL3370 Inner Product Spaces - 3 MOTIVATION AND DEFINITIONS (CONT.) Definition (reminder) A vector space !V over a field !F (e.g. or ) is a set with two operations, sum (!x + y ∈V , for !x, y ∈V ) and scalar multiplication (!λx ∈V , for !x ∈V and !λ ∈F ) such that for all !x, y,z ∈V , !α ,β ∈F : 1. !x + y = y + x (commutativity) 2. !(x + y)+ z = x +( y + z) (associativity) 3. There is a null vector !0∈X such that !0+ x = x 4. !α(x + y)= α x +α y 5. !(α + β)x = α x + βx (distributivity) 6. !(αβ)x = α(βx) (associativity) 7. !0α = 0, !1x = x y A field F is a set with operations + and ⋅ which are: associative and commutative; F has additive and multiplicative identities ( 0 and 1); every a ∈ F has an λx additive inverse ( −a) and, iF a ≠0, a multiplicative inverse x x + y too ( a−1 ); and ⋅ is distributive with respect to + : a⋅(b+c)= a⋅b+a⋅c for all a,b,c ∈ F x EL3370 Inner Product Spaces - 4 MOTIVATION AND DEFINITIONS (CONT.) Definition (reminder) Let !V be a vector space over !F , α1 ,…,αn ∈ F , x1 ,…,xn ∈V , and X ⊆V - Linear combination of x1 ,…,xn : an expression of the form α1x1 +!+αnxn ∈V (for finite n) - lin X (span of X ): set of all linear combinations of elements of X ⊆V - If for every linear combination α x +!+α x =0 it holds that α =!=α =0, X is linearly 1 1 n n 1 n independent (l.i.). Otherwise, X is linearly dependent (l.d.) - Basis of V : A linearly independent set X ⊆V which spans V (i.e., lin X =V ) - dimV (dimension of V ): number of elements of some basis of V . (All bases of V have the same number of elements.) - If dimV <∞, V is a finite-dimensional vector space (obs: V is not necessarily finite!) EL3370 Inner Product Spaces - 5 MOTIVATION AND DEFINITIONS (CONT.) Definition (inner products) An inner product (scalar product) on a vector space !V over is a mapping !( ⋅ ,⋅): V ×V → such that for all !x, y,z ∈V and λ ∈: 1. !(x, y)= ( y,x) 2. !(λx, y)= λ(x, y) 3. !(x + y,z)= (x,z)+( y,z) 4. !(x,x)> 0 when !x ≠ 0 !(V ,( ⋅ ,⋅)) is an inner product space (pre-Hilbert space) Examples 1. Complex vector space !C[0,1]:= { f : [0,1]→ : f !continuous}, with point-wise addition and scalar multiplication (!( f + g)(t)= f (t)+ g(t), !(λ f )(t)= λ f (t) for !f , g ∈C[0,1], λ ∈ 1 and t ∈[0,1]), and inner product ( f , g)= f (t)g(t)dt ! ! ∫0 m×n H 2. Space ! of !m× n complex matrices, with inner product !(A,B)= tr(AB ) EL3370 Inner Product Spaces - 6 MOTIVATION AND DEFINITIONS (CONT.) Proof for Example 1: Since !C[0,1] is a complex vector space (exercise!), we need to verify that !( ⋅ ,⋅) satisfies the axioms of an inner product. Let !f , g,h∈C[0,1] and λ ∈ : 1 1 1. ( f , g)= f (t)g(t)dt = g(t)f (t)dt = (g, f ) ! ∫0 ∫0 1 1 2. (λ f , g)= λ f (t)g(t)dt = λ f (t)g(t)dt = λ( f , g) ! ∫0 ∫0 1 1 1 3. ( f + g,h)= [ f (t)+ g(t)]h(t)dt = f (t)h(t)dt + g(t)h(t)dt = ( f ,h)+(g,h) ! ∫0 ∫0 ∫0 2 2 4. If f ≠ 0, then there is a t ∈[0,1] such that f (t ) =:l ≠ 0. Since f is continuous, there is ! ! 0 ! 0 ! 2 an ε > 0 such that f (t) > l 2 whenever t −t < ε ! ! ! 0 Therefore, 2✏ 1 2 l ( f , f )= f (t) dt ∫0 2 ≥ f (t) dt t [0,1]: t t l/2 2 ∫{ ∈ − 0 <ε} f | | ! ≥ εl 2> 0 ■ 0 t0 1 EL3370 Inner Product Spaces - 7 MOTIVATION AND DEFINITIONS (CONT.) Theorem For every λ ∈ and !x, y,z in an inner product space !V , (i) !(x, y + z)= (x, y)+(x,z) (ii) !(x,λ y)= λ(x, y) (iii) !(x,0)= (0,x)= 0 (iv) If !(x,z)= ( y,z) for all !z ∈V , then !x = y Proof (i) By deFinition: !(x, y + z)= ( y + z,x)= ( y,x)+(z,x)= (x, y)+(x,z) (ii) Similar to (i) (iii) Notice that !(x,0)= (x,0 y), and use (ii) (iv) Since !(x,z)= ( y,z), then !(x − y,z)= 0. Since this holds For every !z , take !z = x − y , which gives !(x − y,x − y)= 0. By the last axiom of an inner product, this implies !x − y = 0 ■ EL3370 Inner Product Spaces - 8 INNER PRODUCT SPACES AS NORMED SPACES Idea: Inner products ⇒ lengths (norms) ⇒ distances (metrics) n T T Example: In ! , !(x, y)= x y ⇒ length!= x = x x = (x,x) ⇒ distance!= x − y Definition In an inner product space !V , the norm of a vector !x ∈V is ! x := (x,x) Examples 2 2 n 1. For x = (x1 ,…,xn )∈ : x = x1 ++ xn ! ! 1 2 2. For f ∈C[0,1]: f := f (t) dt ! ! ∫0 EL3370 Inner Product Spaces - 9 INNER PRODUCT SPACES AS NORMED SPACES (CONT.) Theorem. For every !x, y in an inner product space !V , and λ ∈ : (i) ! x ≥ 0, and ! x = 0 iff !x = 0 (ii) ! λx = λ x (iii) !(x, y) ≤ x y , with equality iff !x = α y for some α ∈ (Cauchy-Schwarz inequality) (iv) ! x + y ≤ x + y (triangle inequality) Proof. (i) Direct from last axiom of an inner product (ii) ! λx = (λx,λx) = λ(x,λx) = λ (x,x) = λ x 2 2 2 (iii) For every α ∈ : !0≤(x −α y,x −α y)= x −2Re{α(x, y)}+ α y 2 2 2 Take !α = tu, where !t ∈ and !u = exp( jarg(x, y)), which gives !0≤ x −2t (x, y) +t y 2 2 2 The minimum of this quadratic expression is ! x − (x, y) y , which must be non- negative. Furthermore, this is zero iff !x −α y = 0 for some α ∈ (iv) By (iii), 2 2 2 2 2 2 2 2 x 2Re(x, y) y x 2(x, y) y x 2 x y y x y ■ ! x + y ≤ + + ≤ + + ≤ + + = ()+ EL3370 Inner Product Spaces - 10 INNER PRODUCT SPACES AS NORMED SPACES (CONT.) Applications of Cauchy-Schwarz inequality Angle between vectors x (x, y) cosθ := x y ! ✓ y Probability 2 Let !V be an inner product space of zero mean real random variables !x with !E{x } < ∞, and inner product !(x, y):= E{xy} = cov(x, y). Then Cauchy-Schwarz inequality implies 2 2 2 2 !cov(x, y) = (x, y) ≤ x y = var(x)var( y) Exercise: Prove that the operations in l are well defined ! 2 EL3370 Inner Product Spaces - 11 INNER PRODUCT SPACES AS NORMED SPACES (CONT.) Applications of Cauchy-Schwarz inequality (cont.) Theorem. In an inner product space !V , the inner product is a continuous function, i.e., for every sequences {x },{ y } such that x → x and y → y , we have (x , y )→(x, y) ! n n ! n ! n ! n n x3 x2 Proof. By Cauchy-Schwarz, xN x1 xN+1 (x, y)−(xn , yn ) = (x, y)−(xn , y)+(xn , y)−(xn , yn ) 0 x ≤ (x − xn , y) + (xn , y − yn ) ball B(x, 1) ≤ y x − x + x y − y ! n n n Since {x } is convergent, it is also bounded (i.e., there is an !M > 0 such that x ≤ M for all ! n ! n n∈); indeed, since there is an !N > 0 such that x − x <1 (say) for !n > N , so ! ! n x = x + x − x ≤ x + x − x < x +1, we can take M = max{ x ,…, x , x +1}. Then, ! n n n ! 1 N given ε > 0, there is an N > 0 such that for n > N , x − x < ε 2 y and y − y < ε 2M , so ! ! 0 ! 0 ! n ! n (x, y)−(x , y ) ≤ y ε 2 y + M ε 2M = ε ■ ! n n () () EL3370 Inner Product Spaces - 12 INNER PRODUCT SPACES AS NORMED SPACES (CONT.) Theorem (Parallelogram Law) Let !x, y be elements of an inner product space. Then, 2 2 2 2 ! x + y + x − y = 2 x +2 y 2 2 2 Proof.