Vector spaces Paul Schrimpf Normed vector spaces Examples Inner product spaces Useful inequalities Vector spaces Projections Row, column, and null space Row space Paul Schrimpf Column space Null space Applications UBC Portfolio analysis Economics 526 First and second welfare theorems Lines, planes, and hyperplanes September 23, 2013 Vector spaces Paul Schrimpf Overview Normed vector spaces Examples Inner product spaces Useful inequalities Projections Row, column, and null space 3 n Row space • Main idea: take intuition from R and apply to R and Column space Null space other vector spaces Applications Portfolio analysis First and second welfare theorems Lines, planes, and hyperplanes Vector spaces Paul Schrimpf 1. Normed vector spaces Normed Examples vector spaces Examples Inner product 2. Inner product spaces spaces Useful inequalities Useful inequalities Projections Row, column, 3. and null space Projections Row space Column space Null space 4. Row, column, and null space Applications Row space Portfolio analysis First and second welfare theorems Column space Lines, planes, and hyperplanes Null space 5. Applications Portfolio analysis First and second welfare theorems Vector spaces Paul Schrimpf Normed vector spaces Examples Inner product spaces Useful inequalities Projections Section 1 Row, column, and null space Row space Column space Normed vector spaces Null space Applications Portfolio analysis First and second welfare theorems Lines, planes, and hyperplanes Vector spaces Paul Schrimpf Normed vector spaces Normed vector spaces • Measure length or distance Examples Inner product spaces Definition Useful inequalities F · ∥·∥ Projections A normed vector space, (V; ; +; ; ), is a vector space F Row, column, with a function, called the norm, from V to and denoted by and null space k k Row space v with the following properties: Column space Null space 1. kvk ≥ 0 and kvk = 0 iff v = 0, Applications k k j j k k 2 F Portfolio analysis 2. αv = α v for all α , First and second welfare theorems 3. The triangle inequality holds: Lines, planes, and hyperplanes kv1 + v2k ≤ kv1k + kv2k for all v1; v2 2 V. • Shortest distance between two points is a straight line Vector spaces Paul Schrimpf Examples Normed vector spaces Examples Inner product Example spaces 3 Useful inequalities R is a normed vector space with norm Projections q Row, column, k k 2 2 2 and null space x = x1 + x2 + x3 : Row space Column space Null space This norm is exactly how we usually measure distance. For Applications Portfolio analysis this reason, it is called the Euclidean norm. First and second n welfare theorems More generally, for any n, R , is a normed vector space with Lines, planes, and hyperplanes norm v u uXn k k t 2 x = xi : i=1 Vector spaces Paul Schrimpf Examples Normed vector spaces Examples Inner product spaces Useful inequalities Example Projections Rn Row, column, with the norm and null space ! Row space p 1=p Column space X Null space k k j jp x p = xi Applications i=1 Portfolio analysis First and second welfare theorems 1 Lines, planes, and for p 2 [1; 1] is a normed vector space. This norm is called hyperplanes the p-norm. 1 Where kxk1 = max1≤i≤n jxi j Vector spaces Paul Schrimpf Examples Normed vector spaces Examples Inner product spaces Useful inequalities Example Projections Lp(0; 1) with p-norm Row, column, and null space ! Z 1=p Row space 1 Column space p Null space k k j j f p = f(x) dx Applications 0 Portfolio analysis First and second welfare theorems Lp Lines, planes, and is a normed vector space. Moreover, (0; 1) is a different hyperplanes space for different p. For example, 1 62 Lp(0; 1), but x1=p 1 2 Lq(0; 1) for q < p. x1=p Vector spaces Paul Schrimpf Normed vector spaces Examples Inner product spaces Useful inequalities Projections Section 2 Row, column, and null space Row space Column space Inner product spaces Null space Applications Portfolio analysis First and second welfare theorems Lines, planes, and hyperplanes Vector spaces Paul Schrimpf Inner product spaces Normed vector spaces • Measure angles Examples Inner product spaces Definition Useful inequalities A real inner product space is a vector space over the field Projections R with an additional operation called the inner product that Row, column, and null space is function from V × V to R. We denote the inner product of Row space 2 h i Column space v1; v2 V by v1; v2 . It has the following properties: Null space h i h i Applications 1. Symmetry: v1; v2 = v2; v1 Portfolio analysis First and second 2. Linear: hav + bv ; v i = a hv ; v i + b hv ; v i for welfare theorems 1 2 3 1 3 2 3 Lines, planes, and 2 R hyperplanes a; b 3. Positive definite: hv; vi ≥ 0 and equals 0 iff v = 0. • Inner product space is also a normed vector space p kxk = hx; xi: Vector spaces Paul Schrimpf Example Normed vector spaces Examples Inner product spaces • Rn Useful inequalities with the dot product, Projections Xn Row, column, T and null space hx; yi = x · y = xi yi = x y Row space Column space i=1 Null space Applications • Norm induced by the inner product is the Euclidean Portfolio analysis First and second norm v welfare theorems u Lines, planes, and n hyperplanes p uX k k h i t 2 x = x; x = xi i=1 Vector spaces Paul Schrimpf Measuring angles Normed vector spaces Examples • Inner product spaces 2 Useful inequalities kx + yk = hx + y; x + yi Projections = hx; xi + 2 hx; yi + hy; yi Row, column, and null space Row space • Rn Column space In with the Euclidean norm when x and y are at right Null space angles to one other, hx; yi = 0, and we have the Applications Portfolio analysis Pythagorean theorem: First and second welfare theorems Lines, planes, and 2 2 2 hyperplanes kx + yk = kxk + kyk : Definition Let x; y 2 V, an inner product space. x and y are orthogonal iff hx; yi = 0. Vector spaces Paul Schrimpf Measuring angles Normed Theorem vector spaces n Examples Let u; v 2 R , then the angle between them is Inner product spaces − hu; vi Useful inequalities θ = cos 1 : Projections kuk kvk Row, column, and null space Row space Column space Proof. Null space Draw picture. Applications k k Portfolio analysis tv First and second cos θ = : welfare theorems kuk Lines, planes, and hyperplanes Use Pythagorean theorem, kuk2 = ktvk2 + ku − tvk2 kuk2 =t2 kvk2 + kuk2 − 2t hu; vi + t2 kvk2 2t hu; vi =2t2 kvk2 Vector spaces Paul Schrimpf Useful inequalities Normed • Triangle inequality vector spaces Examples Theorem (Reverse triangle inequality) Inner product spaces Let V be a normed vector space and x; y 2 V. Then Useful inequalities Projections jkxk − kykj ≤ kx − yk : Row, column, and null space Row space Column space Null space Proof. Applications By the usual triangle inequality, Portfolio analysis First and second welfare theorems Lines, planes, and kxk + kx − yk ≥ kyk hyperplanes kx − yk ≥ kyk − kxk and kyk + ky − xk ≥ kxk ky − xk ≥ kxk − kyk : Vector spaces Paul Schrimpf Useful inequalities Normed vector spaces Theorem (Cauchy-Schwarz inequality ) Examples 2 Inner product Let V be an inner product space and let u; v V. Then, spaces Useful inequalities jh ij ≤ k k k k Projections u; v u v : Row, column, and null space Row space Column space Proof. Null space hu;vi Setup as before, we can show that t = 2 . Now, let Applications kvk Portfolio analysis z = u − tv. By the Pythagorean theorem, First and second welfare theorems Lines, planes, and 2 2 2 hyperplanes kuk = ktvk + kzk hu; vi2 = + kzk2 kvk2 h i2 kzk2 ≥ 0, so kuk2 ≥ u;v kuk kvk ≥ j hu; vi j: kvk2 Vector spaces Paul Schrimpf Normed vector spaces Examples Inner product spaces Useful inequalities Projections Section 3 Row, column, and null space Row space Column space Projections Null space Applications Portfolio analysis First and second welfare theorems Lines, planes, and hyperplanes Vector spaces Paul Schrimpf Projections Normed vector spaces Examples Inner product Definition spaces Let V be an inner product space and x; y 2 V. The Useful inequalities Projections projection of y onto x is Row, column, and null space hy; xi Row space Pxy = x: Column space k k2 Null space x Applications Portfolio analysis More generally, the projection of y onto a finite set First and second welfare theorems fx ; x ; :::; x g is Lines, planes, and 1 2 k hyperplanes Xk P P k y = P j−1 y: fxj gj =1 − xj i=1 Pxi xj j=1 Vector spaces Paul Schrimpf Projections Normed vector spaces Examples Inner product spaces Definition Useful inequalities More generally still, if X ⊆ V is a linear subspace, then the Projections projection of y onto X is Row, column, and null space Row space PX y = Pf gk y Column space bj j=1 Null space Applications 2 Portfolio analysis where bj X and b1; :::; bk span X. First and second welfare theorems Finally, if Y ⊆ V the projection of Y onto X is just the set Lines, planes, and hyperplanes consisting of the projection of each element of y onto X, i.e. PX Y = fPX y : y 2 Yg: Vector spaces Paul Schrimpf Projections Normed Lemma vector spaces Examples Any projection is an idempotent linear transformation. Inner product spaces Proof. Useful inequalities Projections • Verify that projections have the two properties required Row, column, for them to be linear transformations. and null space Row space • Column space Show that projections are idempotent. Null space ! Applications hx; yi Portfolio analysis P (P y) =P x First and second x x x 2 welfare theorems kxk Lines, planes, and hyperplanes D E hx;yi x; k k2 x = x x kxk2 hx; yi hx; xi = x kxk2 kxk2 =Pxy: Vector spaces Paul Schrimpf Normed vector spaces Examples Inner product spaces Useful inequalities Projections Section 4 Row, column, and null space Row space Column space Row, column, and null space Null space Applications Portfolio analysis First and second welfare theorems Lines, planes, and hyperplanes Vector spaces Paul Schrimpf .