Solutions Block 3: Selected Topics in Linear Algebra
Unit 7: The Dot (inner) Product
3.7.1
In our discussion of the game of mathematics at the start of this course, we indicated that one often tries to abstract the crucial aspects of a well-known system in order to see what inescapable consequences follow from the most "reasonable" set of axioms. With this idea in mind, let .us observe mat as of this moment, we have not required our generalized vector space to have the equivalent of a "dot" product defined on it. That is, we have talked about bases of a vector space and about linear transforma- tions, but we have not required that there be defined an operation which allows us to assign to each pair of vectors a number.
Thus, if we were interested in developing the anatomy of a vector space still further, it would next be reasonable to define that part of the structure which is characterized by what the dot product does for ordinary 2- and 3-dimensional vector spaces.
What mathematicians first noticed in this respect was the import- ance of linearity among the various attributes of the dot product. For this reason, they insisted that a rule that assigned to ordered pairs of vectors a real number not even be considered an "inner" (dot) product unless it possess the property of linearity. With this in mind, they defined a bilinear* function on a vector space to be any mapping
such that for a, 6, YEV and c E R
*We say bilinear because the domain of f is ordered pairs.
**Recall that by A x B we mean {(a,b) :a€A, ~EB)). Hence S x S denotes the set of all ordered pairs of elements in S. Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.1 continued
Note #1 In keeping with the notation of the usual dot product, one often writes a-B rather than f(a,B). With this in mind, (1) becomes rewritten as
1. (a + 8) *y = a-y+ Boy 2. a. (6 + y) = a-y + a-y 3. a- (cB) = (ca) *B = c(a B
Note #2 It is crucial to remember that a-B is 2 number, not a member of v. Note #3 Later we shall add the requirement that a-B= B-a , but for now this property is not imposed on our definition of a bilinear function. For this reason 1, and 2, are not redundant.
In other words, if we assume that a-B = B-a for all a, BEV, then
a (B + y) = a-6 + a-y .
Since, however, we do not assume a-B = 6-a , we cannot derive (2) from (1).
Note #4 There are, in general, many different ways (usually infinitely many ways) to define f :V x V + R such that property (1) is obeyed. (We shall discuss this in more detail later). Thus, when we talk about a.6 we are assuming that we have chosen one particular bilinear function. Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.1 continued
With this as background, we now try to explore why the linear properties are so important. What we shall show is that many of the usual structural properties of a dot product follow just by linearity. In particular: + a. w O* = a -(a + 6) =a.d+a06.
Since a ' 8 is, in any event, a number (i.e., a member of R) - let us denote it by b. We then conclude from (2) that
and since b is a number, we conclude by elementary arithmetic from (3) that b = 0. Thus, we have shown that
In summary, we have shown in part (a) that the usual property a 6 = 0 holds for any bilinear function. b. Given,
(a3a3 + a a ), we may treat a3a3 + a4a4 as a (alal + a 2 a 2 ) 4 4 single vector whereupon we may use 1. to conclude that
= (alal + a2a2) -y [where y = a3a3 + a4a4j
and this by 3. equals
+ *We have resorted to the notation 0 to reinforce the idea that a .B in an operation on vectors not numbers. + **It is not an oversight that we have yritten 0 rather than 0 on the right side of (4). Namely, a. 0 is a number, not a vector. Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.1 continued
Replacing y by a3a3 + a4a4, (5) becomes
By 2. we know that
ale (a3 a 3 + a 4 a 4 ) = al- (a3 a 3 + al. (a4a4),
and this, in turn, by 3. is
Similarly
From (7) and (8) we see that (6) may be replaced by
Since equation (9) involves only numbers*, we may use the rules of ordinary arithmetic to replace (9) by
In summary, then, provided that we keep the order "straight" (i.e. is not necessarily the same as a4-al), just as , al' a4 we had to do in our study of the cross-product, we see from (10) that the linearity of the dot product is what allows us to treat it as we treat the analogous expressions of numerical arithmetic.
"That is, while a a 'a3, and a4 are vectors, a a , etc. are numbers. That ist'w%y we have parenthesized a, la,, 'etc., namely L L to emphasize that a1 a2 is number. S.3.7.4 Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.2
a. Once again we restrict our attention to n = 2 for computational simplicity, but our results hold for all n. More importantly, the technique we shall use here is the same one that is used in the higher dimensional cases.
Keep in mind that our main aim in this exercise is to show that a particular bilinear function is completely determined once we know its effect on each pair of basis elements.
So suppose that V = [ul. u21 where {ul, u21is an arbitrarily chosen basis for V. Now let a and B be arbitrary elements of V. Say,
From (1) and part (b) of the previous exercise we conclude that
Since x ,x ,y ,y are known once a and B are given, we see from 1212 (2) that a.B is determined once the values of ul-ul, u~-u~~u2-ulI and u2-u2 are fixed.
Note #1 With alittleinsight, equation (2) gives us a hint as to why our matrix coding system is used in yet another context in vector spaces. Namely, if we let
-P x = [xl ,xzl and t =
we see that equation (2) says that Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.2 continued
As a check on the equivalence of (2) and (2'), we have e .-.
and it is easily seen that the number named by (3) is the same as that named by (2). Note #2 The converse of Note #1is also true. Namely, suppose we let
denote any 2 by 2 matrix. We may now use A to induce a bilinear function f on V x V. Namely, if V = [u1,u21 , we define f as
- - - u2-u1 - a21; and u2*u2 = a22.
We then use (2) to extend the definition to all of V.
This is the essence of part (b) wherein we illustrate our last remark more concretely. b. Let V = [u1,u21, then if a = (3,2) [i.e., 3ul + 2u21 and B = (1,4), then Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.2 continued
In "long hand" our matrix A codes the bilinear function defined by
ul.ul = 1, u -u = -1, u2-u1 = -5, and u .u = 6. 12 2 2
We would then obtain
which checks with (4) . Note #3
What we have essentially shown is that if dim V = n (although we used n = 2) then there are as many bilinear functions which can be defined on V (relative to a fixed basic { ulr...,un } as there are n by n matrices. Namely, if A = [aij] is any n by n matrix, we define a bilinear function by ui-u = a j ij whereupon
is the bilinear function induced by A. Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.2 continued
Note #4 Since there are many different bases for the same vector, two different n by n matrices may code the same bilinear function but with respect to different bases. When we become concerned with this problem we have yet another situation in which we must define what we mean for two matrices to be equivalent. That is, in the same way that we have identified different matrices as being equivalent (in this case, called similar) if they denote the same linear transformation of V but with respect to (possibly) different bases, we might want to identify different matrices as being equivalent if they define the same dot product. Certainly these two definitions of "equivalent" need not be the same. That is, two matrices may code the same linear transformation but not the same dot product.
Hopefully, our treatment in the first two exercises makes it clear that one does not need the bilinear function to be commutative in order for it to have a nice structure. We observe, however, that the "usual" dot product does obey the commutative rule. That is. for any "arrows" and z, g z = g. Thus, to capture, even more, the flavor of the ordinary dot product we add another property to the bilinear function f, namely that f(a ,B) = f(B ,a); or in the Language of the dot product, a-B = B-a .
We call the resulting bilinear form a symmetric bilinear form, and we notice that nothing new happens that didn't already occur in our previous treatment except now all of our matrices which represent the dot product are also symmetric. In other words, for a symmetric bilinear function we know that ulWu2= u2*u1, and even more generally that uiau = u *u Hence the matrix j j i'
is a symmetric matrix (that is, we get the same matrix if we interchange rows and columns). In any event, it is the study
S.3.7.8 Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
of symmetric bilinear functions that makes the study of symmetric matrices especially important. a. If we assume that V = [u1,u21 then the symmetric matrix
A=s ]
codes the symmetric bilinear function defined by
We then have that
Had we wished to arrive at (3) using matrix notation, we would have
which checks with (3). Solutions, Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued b. We know from our geometric treatment of the dot product earlier in our course that the length of the projection of v2 on vl is given by
so thatthe vector projection of v2 onto v1 is given by
Pictorially,
(Figure 1j Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
We may now divide vl and v2* by their magnitudes to determine an orthonormal basis for the space spanned by vl and v2.
Before proceeding to the actual illustration of this exercise, let us observe that while it was convenient to use geometry to construct v2*, it was not necessary to do so. We could have done the same thing algebraically (axiomatically). More importantly, the algebraic approach remains valid for n 2 3 while the geometric approach doesn 't . The algebraic determination of v2* comes from observing that + - OP = xvl (where x is to be determined), that xvl + v2* = v (or v2* = v2 - xvl) and that v2* v1 = 0. Namely,
so that v2* must be given by
which agrees with the geometric situation.
Turning now to our specific exercise, we have
Hence, Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
Then,
Pictorially, z
(Figure 2)
+ + v2* is the vector projection of OP2 onto OP1.
The space spanned by vl and v2 is the plane determined by 0, P1, and P2. Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
As a check that vl and v2* is an orthogonal basis for the space spanned by vl and v2, we need only check that v2* belongs to this space, and vl v2* = 0.
That vl v = 0 may be checked at once from (6). In particular, 2 *
' . - (11,29, -23) v1 v2* = (1,2,3) 7
As to whether v2* belongs to the space spanned by vl and v2, we need simply row-reduce the matrix ;t 1 J v1 v2 -- c - -
to obtain
from which we conclude that S (vlrv2) = {xl (lr0,19) + x2 (0,1,-8) 1
orI
{ (x1tx2, 19x1 -8~~)I .
In particular,
++z = -23. 1 Hence, (11,29, -23) ES (vl,v2) Therefore, v2* = -(Ill29 ,-23) ES . 7 (v11v2) * Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
Note: Do not let the fact that we are now dealing with dot products make you forget that one uses the row-reduced matrix technique quite apart from any knowledge of a dot product. In particular, we analyze the space S(v1,~2) in the same way as we would have in the first three units of this Block. What we should be beginning to feel now is the number of different ways in which matrix notation is used to code completely different aspects of vector spaces.
To conclude this part of the exercise we need only divide both v1 and v2* by their magnitudes and we have obtained the desired orthonormal basis. To this end we have:
= Jv2* . v2* = - dl21 + 841 + 529 v2* 7 = ; .
Hence, we let
Then,
where u ul = u2 u2 = 1 and ul u2 = 0. S(v1,v2) = [u1,u21 1 - c. At first glance it might seem that part (b) was an interruption of our train of thought in going from part (a) to part (c), but this is hardly the case. Rather what we wanted to show is that the technique used in part (b) which is so obvious from a geometric point of view works, word for word, in the more abstract situations.
In particular, in this exercise we want to express the space V = [ul, u ] described in part (a) by replacing u2 by an 2 u2* = 0. appropriate u2*, where u 1 Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
To this end, we need simply take
That is,
where
Moreover, we see from (10) that
Note #1: As we may see from (ll),the matrix of our symmetric bilinear function relative to the basis {ul*, u2*) is given by
That is, the matrix A of part (a) and the matrix B of (12), code the same symmetric bilinear function but with respect to a different basis.
The matrix B shows us why {ul*, u2*) is a "better" basis than {ul, u at least with respect to the given bilinear function. 21 Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
Namely from (12) we see that
In matrix form, (13) may be derived from
Note #2: We may use (11) to find a matrix P such that PAP^ = B. That is, we may diagonalize a symmetric matrix A by appropriately choosing P. This choice of P comes from the following. 1. We know that ul* ul* = u 1 ul, SO relative to {u1,~21as a basis we may code this piece of information by writing
Since u2* = -2ul + u2, we may represent u2* u2* by
Putting (14) and (15) into a single form we have Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
where
and
Note 83: The procedure used in obtaining the diagonal matrix B from the symmetric matrix A generalizes to n-dimensional space. The strange part, at least in terms of our usual experience, is that the diagonalized matrix may include negative entries as well as zero among its diagonal entries. For example, with respect to (12) , notice that -1 appears on the diagonal. What this means is that if we still want to identify v v with the "length" of v, then some symmetric bilinear functions produce the "disturbing" results that a vector can have an imrnaginary length and that a non-zero vector can have a zero length. While it is understandable that such a result may seem disturbing, it is important that we try to understand the structure of mathe- matics to an extent which allows us to accept such results in a natural way. In essence when we say that we want to think of v v as the square of the magnitude of v, we have to realize that unless our axiomatic approach has captured every pertinent ingredient of the system we are trying to characterize, it is possible that certain concepts may possess strange properties in the generalized, abstract system. This happened to us in Block 1 when we introduced modular arithmetic as an example that obeyed much of the structure of ordinary arithmetic, yet was different enough to cause us some startling results. Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
In any event, let us observe that this exercise shows us that relative to our present axioms, a symmetric bilinear function may define a dot product for whichv v < 0 or v v = 0 even if v # 0.
Note #4: Pursuing the previous note in somewhat more detail, we see from (13) that relative to the basis (ul*, u2*1
That is,
hence
Thus, using (16), we see that if v = xul* + yu2*, then
In other words, the vectors v, which are non-zero scalar multiples of ul* +_ u2*, are distinguished by the fact that they are null vectors (v is called a null vector if v # 0 but v v = 0) We may use (10) to investigate how null vectors look relative to the basis {ul, u2) . Namely, solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
while
Check: ' (-u1 + u2) (-u1 + u2) = u1 ' u1 - 2u1 u2 + U2 ' u2
Note #5: The material discussed in Notes 3 and 4 leads to an application of symmetric bilinear functions in the study of the algebraic topic known as quadratic forms. In n variables a quadratic form is any expression of the type
In the special case n = 2, (17) becomes
The connection between quadratic forms and symmetric bilinear functions should become clear rather quickly once we realize 1 that if we let a12 = b12,then Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
In other words, any equation of the form
2 2 allxl + b12x1x2 + a22~2= m
may be viewed as the vector equation
where v v is defined by the symmetric matrix
Relative to our present exercise, we have that
is equivalent to solving v v = m where v = x u + x2u2. That is,
More concretely if we let m = 0 in (19), then we want to solve the equation
This can be done rather simply here (since there are only two variables) by completing the square to obtain Solutions Block 3: Selected ~opicsin Linear Algebra Unit 7: The Dot (inner) Product
3.7.3 continued
The key computational point is that relative to the basis {ul*, u2*1 equation (20) may be written as
where now v = ylul* + y2u2*.
In summary, the process of replacing the symmetric matrix A by the diagonal matrix B allows us to replace a quadratic form by an equivalent form in which only the square terms are present.
a. Since the matrix
is symmetric, it codes a symmetric bilinear function. In parti- cular, if V = [ulru2], then the matrix A codes the symmetric bilinear function defined by
where we have written (2) in a form which we hope suggests the matrix A given in (1). We then have Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.4 continued
In particular, we see from (3) that if (xllx2) = (ylry2) then
By way of review, the right side of (4) is known as a quadratic form. We may always view
as v ' v where the dot product is defined by the symmetric matrix
with a, b, and c as in (5). b. So far we have that
and
We also know that if full u21 is a basis for V so also is {ul + xu2, u21 where x is any real number. So what we try to do is choose x so that
To solve (7) we see that
so that Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.4 continued
Using (6), we see that (8) becomes
We now replace ul by
and we choose as our new basis for V, Eul*, u21 . From (9)
We also know that
since this is how ul* was chosen.
Check: 4 ul* ' u2 = (ul u ) - -5 2 Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.4 continued
Finally
Hence, from (lo), (ll),and (12) we conclude that the matrix of the given dot product relative to the basis (ul*, u21 is
In other words, if VEV is given by v = ylul* + y u , then 1 2 2 2 v.v=- 5 Y12 + 5y2 and the yly2 term is missing. Again, by way of review, (13) may be obtained from (1) as follows. We know that
and that
(14) and (15) may be combined into the single form Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.4 continued
2 c. What (b) shows is that the equation 3xl + 8x1x2 + 5x22 = m is equivalent to - 5 Y12+5y22=mwherethex'sandy'sarere- lated as follows.
If v = ylul* + y2u2*, then
4 where xl = yl and x2 = y2 - 5 yl. In particular,
Hence,
1 v = y u * + y u * is a null vector ++y2 = + 11 2 2 - F yl
u v is a non zero scalar multiple of ul* + u2*
++ v is a non zero scalar multiple of (ul 4 u2) 1 u - 5- + -5 2 3 ++ v = k'(ul u2) or k(ul - u 1, k an arbitrary non-zero - - 5 2 constant.
Check: Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.4 continued
3.7.5 (Review of the Lecture)
As described in the lecture, the process of finding an orthogonal basis may be extended to an n-dimensional space. The procedure is inductive. Namely, we have already seen that if V = [u1,u2,...,u 1 then we may replace u2 by n
whereupon, V = [ul,u2*, u but now u u2* = 0. Once we ..., n I, 1 know that ul and u2 are orthogonal we use the fact that if u 3 is replaced by u3* where u had the form: 3 *
then u1,u2, and u * span the same space as to u and u 3 ltU2' 3 ' We then try to determine xl and x in such a way that ul 2 . u3* = u2 ' u3* = 0.
As long as neither ul ul nor u u2 = 0, this can always be 2 done. Namely, given that
and that ul u = 0, we may dot both sides of (2) by ul to 2 obtain solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.5 continued
whereupon the condition that ul u3* is to equal 0 allows us to conclude from (3) that
Similarly, if we had dotted both sides of (3) with u2 and set u2 u3* equal to 0, we would have obtained
So, putting (4) and (5) into (2) would yield
Equation (6) represents a generalization of our approach in Exercise 3.7.3. Notice that
is the vector projection of u3 onto ul, while
is the vector projection of u3 onto u The procedure indicated 2 ' in (6) is known as the Gram-Schmidt Orthogonalization Process.
It works as follows. Suppose {ul, u } is an orthogonal set ..., n are all different from and that ul . ul, ..., and u n-1 Un-l zero. Given un, we let
We then dot both sides with uk where k = 1,..., or n - 1 and this yields Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.5 continued
Since uk uk # 0 but uk ui = 0 for i # k, we may set uk u * n equal to 0 in (8) to obtain
Putting the result of (9) into (7) we have that if ui . u - 0 j for i # j; i, j = 1,...,n - 1 then if
Looking at (10) one dimension at a time, notice that the coefficient of uk is the vector projection of u n onto uk. In other words, the coefficient of uk has its numerator equal to (minus) u un and k its denominator equal to uk uk. With this as background, we proceed with the present exercise.
.. With V = [u1,u2, u31, the matrix
defines the symmetric bilinear function given by
u1 ' u1 = 1 u1 ' u2 = 1 u1 . u3
Hence to find an orthogonal basis for V we begin by letting Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.5 continued
and
Hence,
V = [ul*, u2*, u31 and ul* u2* = 0.
Since ul* and u2* are orthogonal, we use the Gram-Schmidt pro- cess to replace u 3 by
u2* u u1 ' u3 u3* = u 3 - (ul** u1 *) ul* - (u2* u2
Using (ll),(13) , and (14) we have
Thus, using (17), we see from (16) that
From (13) , (14) , and (18) we conclude that an orthogonal basis for V is given by Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product 3.7.5 continued
Check:
b. From (17) we already know that ul* - ul* = 1 and u2* u2* = 1. From (18),
Hence, relative to {ul*, u2*, u3* , v = x u x u * x3u3*+ 1 11* + 2 2 +
Therefore, Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.5 continued
C. Given that v = 3ul* + 4u2* + 5u3* we see that equation (20) is obeyed. Hence v v = 0, and, accordingly, v is a null vector. Again using (13), (14), and (18) we have
Check:
Hence, putting these three results into a single matrix equation yields Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.5 continued e. The equation
2xZ2 4xj2 2x x 2x1x3 + 6x2x3 = m X1 + + + 12+
is equivalent to solving
Namely (21) evolves from computing v v = m relative to {u1,u2,u3) and obtaining
(i.e., y = xlul + x2u2 + x3u3) and (22) evolves from solving v v = m relative to {ul*, u2*, u3*) ; namely,
where v - ylul* + y2u2* + y3u3*.
We now define an inner, or a dot, product on a vector space V to be any mapping of ordered pairs of elements of V into the real
numbers, say, f :VXV + R which has the following properties.
For vlr v2, v3€V and C,ER
(Y) f(vl, V2) = f(v 1tV2) (ii) f (v1,v2 + v3) = f(vlIv2) + f (vlIv3) (iii)f (vlr cv2) = cf (vlrv2)
(iv) f(vl,vl) 20; and f(vl,vl) = 0 ++ v1 = 0. Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.6 continued
Rewritten in terms of the dot notation we have:
(i) vl-v2=v2 v1 (v2 v3) = v1 v2 v1 v3 (ii) v1 + + (iii)v 1 (cv2) = c(vl . v2) (iv) v1 v1 ->O; vl. vl=o++v1 =o.
When a symmetric bilinear function satisfies property (iv), the function is called positive definite. From another point of view, what positive definite means is that when we replace the symmetric matrix by the equivalent diagonal matrix, every entry on the diagonal is a positive (real) number.
The crucial point is that once we have a positive definite, symmetric, bilinear function, then we can identify the traditional definition of a dot product [namely the definition which says that anbn] with our "new" (al ,...,an) (bl ,...,bn) = a 11b + ... + definition. More specifically, and we shall illustrate this more concretely in the present exercise, if we have a positive definite, symmetric, bilinear function defined on V (and this usually is abbreviated by saying that we have an inner product defined on V), then we may use the Gram-Schmidt Orthogonalization process to find an orthogonal basis for V. Of course, that much we could do before; but now the fact that uk* uk* is positive for each member of our orthogonal basis, means that
is real. In this case we can let
whereupon V = [wl,...,w ] and {wl, w is not only orthogonal n ..., n 1 but also orthonormal, meaning in addition to wi w = 0 if i # j j that wk wk = 1 for each k = l,...,n. This shall be summarized as a note at the end of this exercise. Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.6 continued
Turning to this exercise we have that
codes the symmetric bilinear function, defined on V = [u1,u2,u31, by
Now just as before we let
and
Then [u ,u 1 = [ul*, u2*1, and ul* u2* = 0- 12 We next let
Now, Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.6 continued
and
Consequently (5) may be rewritten as r 1
Matrix Check Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.6 continued
From (6) we conclude that
where
In particular, (6) tells us that our bilinear form is positive definite since each v = V has the form
whereupon
Finally, if we let Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.6 continued
we may verify that {wl, w2, w3) is an orthonormal bases for V. In particular, relative to {wl, w2, w3} , (xl, x2, x3) (ylf y2,
y3) = x1Y 1 + X2Y2 + X3y3' Check:
Note : This is the crucial exercise if we are to identify our definition of a dot product with that given in the text (for that matter, with that given in most texts). Notice that in the text, one identifies vectors with n-tuples, whereupon the dot product is defined by
The key point is that there are as many different ways of repre- senting an n-dimensional space V in terms of n-tuples as there are ways of choosing a basis for V. What this exercise shows is that if our symmetric bilinear function is positive definite (i.e., a dot product), then we can always construct an ortho- denotes this orthonormal basis, normal basis for V. If ul, ...,u n S. 3.7.37 Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.6 continued
then relative to this basis, our dot product is as given by (7). In other words;
sinceu u = landui u =O if if j. i i j Notice, however, that if all we know is that ul,. ..,un is an arbitrary basis for V, then we are not allowed to conclude that relative to this basis (x~~.-.~x~)(ylI...,yn) = xlyl + ... + xnyn since we do not know that ui u = 1 or that ui u for i j i # j. The key point is that once we know that we have a positive definite, symmetric, bilinear function then relative to this there is an orthonormal basis (which may be found by the Gram-Schmidt ortogonalization process). If we now agree to use this basis, then= may think of the dot product in terms of the usual n-tuple definition.
By the usual use of the cross product we know that ul x u 2 is perpendicular to the plane determined by ul and u2. We also know that -t
On the other hand, Solutions Block 3: Selected Topics in .Linear Algebra Unit 7: The Dot (inner) Product
3.7.7 continued
1 Comparing (1) and (2) we have that -ul - u2 + u3 is parallel 1 to (i.e., is a scalar multiple of) ul x u Hence -u - - u + u 2' 1 2 2 3 is perpendicular to the plane determined by ul and u2. In fact, if we want to make the geometric interpretation complete, our construction of u2* in the lecture was equivalent to taking the component of u2 which was perpendicular to ul. Clearly ul and u2* determined the same plane as did ul and u 2 ' It is worth noting that the usual geometric procedure seems much easier to handle than the more abstract Gram-Schmidt Orthogona- lization Process, but the fact remains that the latter method applies to all vector spaces in which an inner product is defined, while the more intuitive geometric technique applies only to 2- and 3-dimensional space.
You may recall in our first Unit in this Block, we emphasized the fact that there were infinite dimensional vector spaces and that one such space was the set of continuous functions on an interval [a,bl. In this exercise we want to point out that the idea of a dot product and the associated concept of "distance" make sense in this rather abstract model of a vector space. In fact, the study of orthogonal functions (of which the special case of Fourier Series will be discussed in the next and final unit) makes excellent sense in terms of the generalized notion of "distance".
More specifically, we know that since f (x) is continuous on [a,b], jabf (x) dx exists; and we also know that the product of two integrable functions is an integrable function. Consequently, b since both f and g are continuous on [a,b] ,$ f (x)g (x)dx is a well-defined number. Thus, the definition Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.8 continued
is at least a function which maps ordered pairs of continuous functions, defined on [a,b], into the real numbers; and accord- ingly the definition given above is at least eligible to be considered as a possible dot product. Our aim in this exercise is to show that indeed (1) does define a dot product; and we do this by showing that the definition given in (1) satisfies properties (i), (ii), (iii), and (iv) required of any dot product.
Given that
then
(ii)(cf) g = [[cf (x, Ig(x)dx
(iii) f. g= 6"£(x)g(x)dx = P.(XI f (XI - Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.8 continued
2 2 and since [f(x)1 2 0 for all x, k[f(x)1 dx -> 0 for all x and 2 $[£(x)~ dx= 0 ++f(x) O* ++f = 0; f f -> 0 and f. f = 0 ++ f = 0. Hence, since (i), (ii), (iii)and (iv) are all obeyed f g, defined by f g = $f (~)~(x)dxis a positive definite, symmetric, bilinear function. In other words, it is an inner product.
Note : Once we have a dot product defined on a vector space, it makes sense to talk about orthogonality. In particular, suppose that fl,...,fn are any linearly independent functions which are con- tinuous on [a,bl and that for i # j, fi ' f = 0. That is, for j i#jI
We then have a simple way of computing the coefficients cl, ..., and cn in an expression such as
g(x) = clfl(x) + ... + cn f n (x).
Indeed we need only employ the usual technique of dotting both sides of (3) with fk(x) (for any k = 1,2,3,. ..,n) in order to find each ck. In particular, this leads to
Each term on the right side of (4) is zero except for ckfk fk, since fi f = 0 whenever i # j. j If we rewrite (4) in terms of definition (l),we obtain
*Here we use the fact that f is continuous for otherwise we could allow f (x) to be unequal to 0 at, for example, a finite number of points in [a,b] and yet Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.8 continued
from which we conclude that
[Notice that fk # 0 since f . ..,£ 1 is linearly independent. 2 hence by (iv) , $[fk (x)1 dx # 0 and, consequently (6) is well- defined .] If also we assume that the f's are normalized, i.e., f f = 2 db[f (XI I dx = l*, then (6) may be further simplified to read
Using the result of (7) in (3) we obtain
As is usually the case, very few serious problems arise as long as we look at finite linear combinations. The deep study begins when we assume that we are dealing with an infinite sets of functions defined and continuous on an interval [arb]. For now, just as in our study of power series in Part 1 of this course, we must ask the question of whether every continuous function defined on [a,b] can be represented as a convergent series in
*If f . f # 1, we are in no great trouble. Namely, if f f = k $ 1 (k $ 0), then since f f 2 0, k 2 0, & is real. We then replace f by f/ whereupon
11 & is then called a weighting factor for f. Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.8 continued
which the series is an infinite linear combination of elements of our set S. Once we have answered this question (hopefully, in the affirmative) we must then ask whether the process of dotting term by term as we did in this exercise is valid. In other words, as usual, such statements as "the integral of a sum is the sum of the integrals" depends on the fact that our sum involves only a finite number of terms.
The discussion hinted at here forms the foundation of such topics as representations by Fourier series. We shall not go into the theory of Fourier series in this course, but we shall, in the next unit, talk about a few properties of Fourier series.
3.7.9 (optional) a. Suppose
then
Since {ul,...,un) are orthogonal we have that ul u = 0 for j j = 2,. ..,n. This, coupled with the fact that ul 0 = 0, reduces (21 to
Now, since c and ul u are real numbers, we conclude from (3) 1 1 that either cl = 0 or ul u1 = 0.
But ul ul = 0 ++ u1 = 0 (i.e., a dot product is positive definite). Hence, if ul # 0, then cl = 0. We may repeat this procedure by dotting both sides of (1)with u2, u3,..., or un; and we conclude that if (ul, ...,un) is any set of non-zero orthogonal vectors, then clul + ... =Oc+c = ..' = C = 0. + 'nun n In other words, any set of non-zero orthogonal vectors is linearly independent. This does not mean that one needs the concept of a dot product to study linear independence but rather Solutions Block 3:. Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.9 continued
that once one knows about a dot product, there are easier ways of testing for linear independence.
Along these same lines, notice how we used this idea when we -t 3 dealt with 1, $, and k components in E . Quite generally, if
where ul, ..., and un are orthogonal, we may compute cl by dotting both sides of (4) with ul. Since
we deduce from (4) that
If we now assume, in addition, that {ul, ...,un} is orthonormal, then ul ul = 1, whereupon (5) yields
What equation (6) corroborates is our usual property of 3-dimensional orthonormal coordinate systems. Namely, if Iu1,...,un} is orthonormal and if v is any linear combination of ul,..., and un; then we find the ui-component of v merely by dotting v with ui. b. In this part of the exercise we are trying to show how the concept of an orthogonal complement extends the idea of perpendi- cularity as studied in the lower dimensional cases. For example, in 2-space we know that the 1-dimensional subspaces are lines. Any two non-parallel lines span the plane, but we can always pick our lines to be perpendicular. In a similar way, given a plane W in 3-space then any vector in 3-space is the sum of two vectors, one of which lies in W and the other of which doesn't. We can always choose the second vector to be perpendi- cular to W.
Our point is that this idea extends to any vector space on which a dot product is defined. The basic idea is as follows: Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.9 continued
(i) Suppose that W is any subspace of V. We then define a new set W by P
W = {VEV:~w = 0 for each we^) P
Clearly, by its very definition, W is a subset of V, but as we P ask you to show in this part of the exercise, W is also a P subspace of V. To show this, all we have to do is prove that the sum of two members of W is also a member of W and that every P PI scalar multiple of an element of W belongs to W P P . Computationally, this is certainly easy enough to do. Namely:
v ,v EW + v w = v2 w = 0 for each weW 12P 1
w + v2 w = 0 for each WEW + v1 .
+ (vl + v2) w = 0 for each WEW
VEW + v w = 0 for each WEW P
+ c(v W) = 0 for each wsW, where c is any scalar
+ (cv) w = 0 for each WEW
(ii) If veW then v w = 0 for each weW. In particular, then, P if v also belongs to W, then by virtue of it being a member of W v . v = 0. Since our dot product is, by definition, positive P definite, v v = 0 + v = 0. Hence, veWnW + v = 0, or P
L- (iii) We now show that V = W @ Wp. That is, each veV may be expressed in one and only one way as a sum of two vectors, one of which is in W and the other of which is perpendicular to W (i.e., in W ). P Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.9 continued
Since WnW = (01 , it is sufficient to show that each VEV may be P written in at least one way in the required form.*
We introduce the Gram Schmidt process to solve our problem. Namely, we begin by constructing an orthonormal basis for W, say {wl,...,w r 1. We may now use the Gram-Schmidt Orthogonalization Process to augment this basis to become a basis for V. That is
= [wl, Wr. vl,... IV I I v ..., n-r
where dim V = n.
The key lies in the fact that W = [vl,. ,v I where v and P .. n-r 1'"" v are as given by (8). To prove this assertion we first notice n-r that it is trivial to show that vl, .... and vn-r all belong to W P ' Namely each of these vectors is by construction orthogonal to each of the vectors wl.. . . , and wr; hence to the space spanned by wl,.... and wr. As a computational review, we have that for each wEW,
Hence, vi E W P'
*Recall that if W1.nW2 = (0) and wl + w = w ' + w ', where wlYw1'~W1 and w2, *EW~. then w = w "and lw = 2w2'. Namely 2 + w2 = w ' + w '2+ w - w ' =lw ' w Hence, - w '&W2 1 2 1 . 1 since w -lw is in w2? Since w12- w '2~we havewhat wl w I= ' 2 - 1 W1 and since W = 0 w J= 0 from which n wZ2; y2 W11 1 w A similar argument shows that w2 = w2'. we conclude that 1 =lwl . Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.9 continued
Conversely if w is any element of W then since W c V, we see from P P P (8) that
Now, since w w = 0 for each WEW, we know that in particular P - ... = w - w, = 0. Thus, for example, if we dot both W~ ' W1- P sides of (9) with wl we obtain
. w = (CW + ... + crwr + alvl+ ... + a v wl P 1 11 n-r n-r
and since {wl. ....wr, vl..., vn-r 1 is orthonormal, we conclude - - from (10) that 0 = cr Similarly, we may show that c2 = ... - Cr- 0, so that by (9) ,
and this shows that Cvl, ...,vn-r 1 span W . This completes our P proof that
3.7.10 (optional)
We have
u4 u1 = 1, u4 u = 2, u4 u3 = 1, U4 - . 2 u4=9J We first let Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.10 continued
Then by the Gram-Schmidt process we have
or from (1),
[Check: u u = 1 1 1* 2 * u1 (u2 - ;?. ul) = u1 . u2 - 7 u, 1 = 1 - ;?.(4) = 1 - 1 = 0.1 With ul* as in (2) and u2* as in (3)' we next define u3* by
U 'U* u -u* 3 l 3 u3* = u - ul* - u*. ul*. ul* u2*. u2* Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.10 continued
~utting~the results of (5), (6) , (7) and (8) into (4) yields
Finally,
2 7 u2* u2* [by (8)] = 4
11 10 u4 u3* = [by (9) 1 u4 (u3 - -27 u1 - 27 u2) 11 10 = u4 u3-- 27 u4 ' u1 - 27 u4 ' 2 11 =1-3(1) - -lo27 (2) Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.10 continued
Putting these results into (10) yields
Summarizing, we have from (21, (31, (91, and (11)
u*=-ul u 2u3 u,, 4 - 2 + + Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.10 continued
Note :
If we write (12) in matrix form we have
Matrix (13) tells us that Solutions Block 3: Selected Topics in Linear Algebra Unit 7: The Dot (inner) Product
3.7.10 continued
Hence, ul*, u2*, u3*, u4* is an orthogonal basis for V; and
This is much neater than the corresponding result using u ,u ,u and u4 as a basis in which case we would also have 123, - had to worry about the terms involving x1y2, y x1y4, x2y1, x 13, X2Y3f X2Y4r X3Y1r X3Y2t X3Y4' X4Y1, X4Y28 and X4Y3- MIT OpenCourseWare http://ocw.mit.edu
Resource: Calculus Revisited: Complex Variables, Differential Equations, and Linear Algebra Prof. Herbert Gross
The following may not correspond to a particular course on MIT OpenCourseWare, but has been provided by the author as an individual learning resource.
For information about citing these materials or our Terms of Use, visit: http://ocw.mit.edu/terms.