INNER PRODUCTS and NORMS Professor Dan A. Simovici UMB Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 1 / 46 1 Norms 2 Inequalities 3 Metric Spaces 4 Norms on Rn 5 Norms for Matrices 6 Inner Products Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 2 / 46 Seminorms and Norms A seminorm on an linear space V is a mapping ν : V −→ R that satisfies the following conditions: ν(x + y ) ≤ ν(x )+ ν(y ) (subadditivity), and ν(axx)= |a|ν(x ) (positive homogeneity), for x ,y ∈ V and a ∈ F . Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 3 / 46 Example A seminorm can be defined on every linear space. Indeed, if B is a basis of V , B = {v i | i ∈ I }, J is a finite subset of I , and x = i∈I xiv i , define νJ (x ) as P 0 if x = 0, νJ (x )= ( j∈J |aj | otherwise for x ∈ V . P Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 4 / 46 If V is a real or complex linear space and ν : V −→ R is a seminorm on V , then ν(x − y ) ≥ |ν(x ) − ν(y )|, for x ,y ∈ V . If p : V −→ R is a seminorm on V , then p(x ) ≥ 0 for x ∈ V . Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 5 / 46 A norm on an F -linear space V is a seminorm ν : V −→ R such that ν(x ) = 0 implies x = 0 for x ∈ V . The pair (V ,ν) is referred to as a normed linear space. Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 6 / 46 Example The set of real-valued continuous functions defined on the interval [−1, 1] is a real linear space. The addition of two such functions f , g, is defined by (f + g)(x)= f (x)+ g(x) for x ∈ [−1, 1]; the multiplication of f by a scalar a ∈ R is (af )(x)= af (x) for x ∈ [−1, 1]. Define ν(f ) = sup{|f (x)| | x ∈ [−1, 1]}. Since |f (x)|≤ ν(f ) and |g(x)|≤ ν(g) for x ∈ [−1, 1]}, it follows that |(f + g)(x)| ≤ |f (x)| + |g(x)|≤ ν(f )+ ν(g). Thus, ν(f + g) ≤ ν(f )+ ν(g). Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 7 / 46 Preliminaries R 1 1 Let p, q ∈ − {0, 1} such that p + q = 1. Then p > 1 if and only if q > 1. Furthermore, one of the numbers p, q belongs to the interval (0, 1) if and only if the other number is negative. R 1 1 Let p, q ∈ − {0, 1} be two numbers such that p + q = 1 and p > 1. Then, for every a, b ∈ R≥0, we have ap bq ab ≤ + , p q − 1 where the equality holds if and only if a = b 1−p . Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 8 / 46 The H¨older Inequality Let a1,..., an and b1,..., bn be 2n nonnegative numbers, and let p and q 1 1 be two numbers such that p + q = 1 and p > 1. We have 1 1 n n p n q p q ai bi ≤ ai · bi . i i ! i ! X=1 X=1 X=1 Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 9 / 46 Theorem Let a1,..., an and b1,..., bn be 2n numbers and let p and q be two 1 1 numbers such that p + q = 1 and p > 1. We have 1 1 n n p n q p q ai bi ≤ |ai | · |bi | . ! ! i=1 i=1 i=1 X X X Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 10 / 46 The Cauchy-Schwarz Inequality for Rn Let a1,..., an and b1,..., bn be 2n real numbers. We have n n n 2 2 ai bi ≤ ai · bi . v v i=1 u i=1 u i=1 X uX uX t t Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 11 / 46 Minkowski Inequality Let a1,..., an and b1,..., bn be 2n nonnegative real numbers. If p ≥ 1, we have 1 1 1 n p n p n p p p p (ai + bi ) ≤ ai + bi . i ! i ! i ! X=1 X=1 X=1 Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 12 / 46 Definition 2 A function d : S −→ R≥0 is a metric if it has the following properties: d(x, y)=0 if and only if x = y for x, y ∈ S; d(x, y)= d(y, x) for x, y ∈ S; d(x, y) ≤ d(x, z)+ d(z, y) for x, y, z ∈ S. The pair (S, d) will be referred to as a metric space. If the first property is replaced by the weaker requirement that d(x, x) = 0 for x ∈ S, then we refer to d as a semimetric Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 13 / 46 Example 2 Let S be a nonempty set. Define the mapping d : S −→ R≥0 by 1 if u 6= v, d(u, v)= (0 otherwise, for x, y ∈ S. Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 14 / 46 Example 2 R Consider the mapping d : (Seqn(S)) −→ ≥0 defined by d(p,q)= |{i | 0 ≤ i ≤ n − 1 and p(i) 6= q(i)}| for all sequences p,q of length n on the set S. Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 15 / 46 Example The Euclidean metric is the mapping n d (x ,y )= (x − y )2. 2 v i i u i=1 uX t Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 16 / 46 Spheres Let (S, d) be a metric space. The closed sphere centered in x ∈ S of radius r is the set Bd (x, r)= {y ∈ S|d(x, y) ≤ r}. The open sphere centered in x ∈ S of radius r is the set Cd (x, r)= {y ∈ S|d(x, y) < r}. Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 17 / 46 Definition n For p ≥ 1, the function νp : R −→ R≥0 defined by 1 n p p νp(x1,..., xn)= |xi | , i ! X=1 n n where x = (x1,..., xn) ∈ R , is a norm on R . Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 18 / 46 Examples The following are norms on Rn: n The mapping ν1 : R −→ R given by ν1(x )= |x1| + |x2| + · · · + |xn|, n for x = (x1,..., xn) ∈ R . n ν∞ : R −→ R≥0 given by ν∞(x ) = max{|xi | | 1 ≤ i ≤ n} n for x = (x1,..., xn) ∈ R . k x k2 is the Euclidean norm Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 19 / 46 Any norm generates a metric Each norm ν : V −→ R≥0 on a real linear space V generates a metric on the set V defined by dν(x ,y )= ν(x − y ) for x ,y ∈ V . Note that ν can be expressed using dν as ν(x )= dν(x ,00) for x ∈ V . For p ≥ 1, dp denotes the metric dνp induced by the norm νp on the linear space Rn known as the Minkowski metric. on Rn. Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 20 / 46 Metrics induced by norms p = 2: the Euclidean metric on Rn is n n d (x ,y )= |x − y |2 = (x − y )2. 2 v i i v i i u i=1 u i=1 uX uX t t p = 1: n d1(x ,y )= |xi − yi |. i X=1 This metric is known also as the city-block metric. The norm ν∞ generates the metric d∞ given by d∞(x ,y ) = max{|xi − yi | | 1 ≤ i ≤ n}, also known as the Chebyshev metric. Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 21 / 46 If x = (x0, x1) and y = (y0, y1), then d2(x ,y ) is the length of the hypotenuse of the right triangle and d1(x ,y ) is the sum of the lengths of the two legs of the triangle. 6 y = (y0, y1) x = (x0, x1) (y0, x1) - Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 22 / 46 Projections on Closed Sets Theorem n n Let U be a closed subset of R such that U 6= ∅ and let x 0 ∈ R − U. Then, there exists x 1 ∈ U such that k x − x 0 k2≥k x 1 − x 0 k2 for every x ∈ U. Let p and q be two positive numbers such that p ≤ q. For every n u ∈ R , we have k u kp≥k u kq. Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 23 / 46 Comparisons between norms Let a1,..., an be n positive numbers. If p and q are two positive numbers such that p ≤ q, then 1 1 p p p q q q a1 + · · · + an ≥ a1 + · · · + an . Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 24 / 46 Spheres Bdp (00, 1) for p =1, 2, ∞ 6 6 6 @ @ - '$- - @ @ (a)&% (b) (c) Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 25 / 46 Matrix Vectorization The (m × n)-vectorization mapping is the mapping vec : Cm×n −→ Rmn defined by a11 . . a m1 . vec(A)= . , a n 1 . . amn obtained by reading A column-wise. Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 26 / 46 Example For the matrix In we have e 1 e 2 vec(In)= . . e n Professor Dan A. Simovici (UMB) INNER PRODUCTS and NORMS 27 / 46 Vectorial Norms of Matrices Let ν be a vector norm on the space Rmn.