11.2MH1 LINEAR ALGEBRA Summary Notes

4 Inner Product Spaces

Inner product is the abstraction to general vector spaces of the familiar idea of the scalar product of two vectors in 2 or 3 . In what follows, keep these key examples in mind. An inner product on a vector space V is a function V V , usually denoted u v u v , satisfying four axioms:

1. u v v u

2. u v w u v u w

3. u v u v

4. v v 0, with v v 0 if and only if v 0.

De®nition A vector space V equipped with an inner product is called an inner product space. Examples There are many examples of inner products on vector spaces. The standard example is the euclidean inner product

u v utv

n on Mn1. Another class of examples are given by integrals, where the vector space V consists of suitable real-valued functions. For example, if V is the space of continuous functions 0 1 , then 1 f g f x g x dx 0 is an inner product on V.

An important property of an inner product is the:

Cauchy-Schwartz Inequality u v 2 u u v v u v V Proof. If u 0 then both sides are 0 and there is nothing to prove. Suppose then that u 0. Since 0 u v u v u u 2 2 u v v v the theory of quadratic equations tells us that

2 u v 2 4 u u v v 0 As in the familiar examples of 2 and 3 , we can use the inner product to de®ne the `length', or norm of a vector as v v v and the `angle' formed (at the origin) by u and v as u v cos u v (This makes sense, since the Cauchy-Schwartz inequality implies that the right hand side lies between 1 and 1.) In particular, u v are orthogonal if they make an angle 2, ie if u v 0. A set of vectors v1 vn is orthogonal if vi v j 0 whenever i j. If in addition each vi has norm 1, ie vi vi 1, then it is orthonormal.

If B v1 vn is an orthogonal basis for V, then the scalars 1 n that appear in the expression of a vector v as a linear combination

1v1 nvn (the coordinates of v with respect to B) can easily be computed as

v vi i vi vi

If, moreover, B is orthonormal (ie vi vi 1 for all i), then this simpli®es to:

i v vi

To form an orthonormal basis from an existing basis u1 un , we use:

Gram-Schmidt Process 1. Set u1 v1 u1

2. Assume that we have constructed an orthonormal basis v1 vk for Span u1 uk , where 0 k n. De®ne

k uk 1 uk 1 uk 1 vi vi i 1

Note that uk 1 0, since uk 1 Span v1 vk . Set

uk 1 vk 1 uk 1

2 3. REPEAT UNTIL k n.

Example Use the Gram-Schmidt process to transform 3 4 5 6 to an orthonormal basis for 2 .

u1 3 4 , u1 5, so u1 3 4 v1 u1 5 5 39 u2 5 6 , so u2 v1 5 , and 39 8 6 u u v 2 2 5 1 25 25 Finally u2 4 3 v2 u2 5 5

Extending orthonormal bases for subspaces

Suppose we have formed an orthonormal basis u1 un 1 for a hyperplane

n H x1 xn 1x1 nxn

n n in . There is a quick way to extend this to an orhtonormal basis for : let vn be the t normal vector 1 n to H. (Then ui vn 0 i.) Set

vn un vn Then u1 un is the desired basis for n. More generally, if U is the solution space of a system of t homogenous equations AX 0, then the column space of A is orthogonal to U, so if B1 B2 are orthonormal bases for U and for the column space of A respectively, then B1 B2 is an orhtonormal basis for n.

Coordinates

Suppose that V is a ®nite dimensional vector space, and B v1 vn is a basis for V. That means that every element v V can be uniquely expressed in the form

v 1v1 nvn

3 with i . The i are called the coordinates of v with respect to the basis B, and the column matrix 1 . X . Mn1 n is called the coordinate matrix or the coordinate vector of v with respect to B. If V is an inner product space, and B is an orthogonal basis, then the coordinates can be readily calculated as: v vi i vi vi If in fact B is orthonormal, then the denominator in this expression is 1, and we get the simpler expression i v vi

Change of basis

Suppose that B v1 vn and B v1 vn are two different bases for V. How do the coordinates of an element v V with respect to B relate to the coordinates of v with respect to B ? Suppose 1 . X . n is the coordinate matrix of v with respect to B, and

1 . Y . n is the coordinate matrix of v with respect to B . Since

v 1v1 nvn it follows that Y 1C1 nCn where Ci is the coordinate matrix of vi with respect to B . If A is the n n matrix with columns C1 Cn, then Y AX

The matrix A is called the transition matrix from B to B . Some obvious properties of transition matrices:

4 1. The transition matrix from B to B is the identity matrix I, for any basis B.

2. If A is the transition matrix from B to B , and A is the transition matrix from B to a third basis B , then the matrix product A A is the transition matrix from B to B . (To see this, suppose X Y Z are the coordinate matrices of some element v V with respect to the three bases B B B . Then Y AX and Z A Y, so Z A A X.)

3. Putting together the two properties above, we see that, if A is the transition matrix from B to B , then A is invertible, and its inverse A 1 is the transition matrix from B to B.

Theorem Let V be an inner product space and let B B be two orthonormal bases for V. Then the transition matrix from B to B is orthogonal, that is At A 1.

Proof. By de®nition, the i j -entry of A is the vi-coordinate of v j, which is vi v j , since B is orthonormal. But B is also orthonormal, so by the same argument this is also the j i -entry of the transition matrix from B to B, ie of A 1. Hence A 1 At, as claimed.

5