These notes closely follow the presentation of the material given in David C. Lay’s textbook Linear Algebra and its Applications (3rd edition). These notes are intended primarily for in-class presentation and should not be regarded as a substitute for thoroughly reading the textbook itself and working through the exercises therein. The Dot Product of Two Vectors in n Suppose that
u1 v1 u and v u2 v2 are two vectors in 2. We define the dot product (also called the inner product or the scalar product) of u and v to be u v uTv. Thus,
T v1 u v u v u1 u2 u1v1 u2v2 . v2 Note that u v is actually a 1 1 matrix. However, we will adopt the convention that u v is a scalar (equal to the single entry in this 1 1 matrix). For example, if 1 7 u and v , 3 3 then u v 17 33 2. Clearly, this definition can be extended to n for any positive integer n:If
u1 v1
u2 v2 u and v
un vn are any two vectors in n, then we define
u v u1v1 u2v2 unvn. When convenient, we will simply write u v uTv (even though, technically, u v is a scalar and uTv is a 1 1 matrix).
1 Example Let 1 3 5 u 6 , v 4 , and w 5 . 8 1 0 Compute u v, v u, 3u v, u 3v, 3u v, u u, u v w, and u v u w.
2 Basic Properties of the Dot Product 1. If u and v are any two vectors in n, then u v v u. 2. If u and v are any two vectors in n and c is any scalar, then cu v u cv cu v. 3. If u, v, and w are any three vectors in n, then u v w u v u w. n 4. If u is any vector in , then u u 0. Also, u u 0 if and only if u 0n.
3 Length and Distance For any vector u n, we define the length (also called the norm or the magnitude)ofu to be u u u . Thus, if
u1
u2 u ,
un then 2 2 2 u u u u1 u2 un .
Example Find the length of the vector 1 u . 5
4 If u and v are any two vectors in n, then we define the distance between u and v to be u v.
Example Find the distance between the vectors 1 3 u and v . 8 9
5 Basic Properties of Length and Distance n 1. If u is any vector in , then u 0. Also, u 0 if and only if u 0n. 2. If u is any vector in n and c is any scalar, then cu |c|u. 3. If u and v are any two vectors in n, then u v v u. 4. If u and v are any two vectors in n, then |u v| uv. (This is called the Cauchy–Schwarz Inequality.) 5. If u and v are any two vectors in n, then u v u v. (This is called the Triangle Inequality.)
6 n A unit vector is a vector of length 1.Ifv is any vector in with v 0n, then it is easy to find a unit vector that points in the same direction as v: In fact, the vector u 1 v v has length 1 and is a positive scalar multiple of v.
Example Find a unit vector that points in the same direction as the vector 1 u 6 . 8
7 Orthogonality of Vectors Two vectors, u and v,inn are said to be orthogonal (or perpendicular) to each other if u v 0. This definition of orthogonality is inspired by what we can visualize in 2 or 3: Two non–zero vectors, u and v,in2 (or 3) are perpendicular to each other if and only if u v u v, and this is true if and only if u v 0.
Example Show that the vectors 9 2 u and v 9 4 2 are orthogonal to each other, and show that the vectors 2 1 u and v 6 4 are not orthogonal to each other.
8 Example Describe the set of all vectors in 2 that are orthogonal to the vector 9 u . 4
9 Example Describe the set of all vectors in 3 that are orthogonal to the vector 4 u 4 . 7
10 Example Let 1 3 W Span 1 , 0 . 6 5 Note that W is a two–dimensional subspace of 3 (a plane in 3 that passes through the origin in 3). Describe the set of all vectors in 3 that are orthogonal to every vector in W.
11 Definition If W is a subspace of n, we define the orthogonal complement of W, denoted by W, to be the set of all vectors in n that are orthogonal to all of the vectors in W.
Theorem Suppose that W is a subspace of n. Then: 1. W is a subspace of n. 2. W W 0n . 3. dimW dimW n and if BW is a basis for W and BW is a basis for W , n then BW BW a basis for .
n Remark If W and U are any two subspaces of such that W U 0n and such that the union of a basis for W and a basis for U is a basis for n, then we say that n is the direct sum of W and U and we write n W U. Thus, the above theorem tells us that if W is any subspace of n, then n W W. We remark that n W U U W but that U W n W U.
Example If 9 W Span , 4 then W is a subspace of 2. In fact, W is a line passing through the origin in 2. The subspace W consists of all vectors
x1 x 2 x2 such that
9x1 4x2 0, or, to write this in a different way,
x1 9 4 0. x2 Since 4 9 4 ~ 1 9 ,
12 we see that 4 4 x1 t 4 x 9 t 9 t . x2 t 1 9 Therefore 4 W Span . 9 Note that 9 4 W W Span , 2. 4 9
Example If 4 W Span 4 , 7 then W is a subspace of 3. In fact, W is a line passing through the origin in 3. The subspace W consists of all vectors
x1 2 x x2
x3
13 such that
4x1 4x2 7x3 0, or, to write this in a different way,
x1
447 x2 0.
x3 Since 7 447 ~ 1 1 4 , we see that 7 7 x1 t 4 s 1 4 1 7
x x2 t t 1 s 0 t 1 s 0 .
x3 s 0 1 0 4 Therefore 1 7 W Span 1 , 0 . 0 4 Note that 4 1 7 W W Span 4 , 1 , 0 3. 7 0 4
Example If 1 3 W Span 1 , 0 , 6 5 then W is a subspace of 3. In fact, W is a plane passing through the origin in 3. The subspace W consists of all vectors
x1 2 x x2
x3 such that
14 x1 x2 6x3 0
3x1 5x3 0 or, to write this in a different way,
x1 1 16 0 x2 . 305 0 x3 Since 5 1 16 10 ~ 3 , 23 305 01 3 we see that 5 5 x1 3 t 3 5 x 23 t 23 t . x2 3 t 3 23 x3 t 1 3 Therefore 5 W Span 23 . 3 Note that 1 3 5 W W Span 1 , 0 , 23 3. 6 5 3
Theorem If A is any m n matrix, then rowA nulA and n rowA nulA. Also, colA nulAT and m colA nulAT . Proof Suppose that r rowA and n nulA. Then r ATx for some vector x m and An 0m. Now note that T T T T T T r n r n A x n x An x An x 0m 0. This shows that every vector in rowA is orthogonal to every vector in nulA. and hence that rowA nulA. Now, suppose that x rowA. Then r x 0 for all vectors r rowA. Thus,
15 r1 r1 x
r2 r2 x Ax x 0m,
rm rm x which shows that x nulA, and hence that rowA nulA. We conclude that rowA nulA and hence that n rowA nulA. To verify the second assertion of the theorem, note that colA rowAT nulAT .
Example In order to illustrate the above theorem, suppose that A is the matrix 26 9 A 051 . 4719 We will show that 3 rowA nulA and that 3 colA nulAT . First, note that 51 26 9 10 10 10051 A ~ 1 ~ 051 01 5 0 51 4719 00 0 000 which shows that 10 0 rowA Span 0 , 5 . 51 1 We also see that nulA consists of all vectors x 3 such that 51 51 x1 10 t 10 51 x 1 t 1 t . x2 5 t 5 2 x3 t 1 10 Therefore, 51 nulA Span 2 . 10 Now, observe that 3 rowA nulA.
16 Next, note that 204 10 2 AT 657 ~ 011 9 119 00 0 which shows that 1 0 colA rowAT Span 0 , 1 2 1 and 2 nulAT Span 1 . 1 We observe that 3 colA nulAT . As a double–check, observe that the row reduction that was performed earlier: 51 26 9 10 10 A ~ 1 051 01 5 4719 00 0 shows that 2 6 colA Span 0 , 5 4 7 from which it can still be observed that 3 colA nulAT .
17 Extending the Idea of Orthogonality to Vector Spaces Other Than n Definition Let V be a vector space. An inner product for V is a binary function , : V such that 1. If u and v are any two vectors in V, then u, vv, u. 2. If u and v are any two vectors in V and c is any scalar, then cu, vu, cvcu, v. 3. If u, v, and w are any three vectors in V, then u, v w u, v u, w. 4. If u is any vector in V, then u, u 0, and u, u 0 if and only if u 0V.
Example The dot product is an inner product for n.
Example For the vector space C0, (consisting of all continuous functions f : , ), we can define an inner product as follows: f,g fxgxdx. To verify that this is indeed an inner product for C0,, note that: 1. If f and g are any two functions in C0,, then f,g fxgxdx gxfxdx g,f. 2. If f and g are any two functions in C0, and c is any scalar, then cf,g cfxgxdx c fxgxdx f,cg fxcgxdx c fxgxdx cf,g c fxgxdx. Thus cf,g f,cg cf,g. 3. If f , g, and h are any three functions in C0,, then f,g h fxgx hxdx fxgx fxhxdx fxgxdx fxhxdx f,g f,h. 4. If f is any function in C0,, then f,f fxfxdx fx2 dx 0
18 (because fx2 0 for all x ,. Also, fx2 dx 0 if and only if fx 0for all x , which means that f,f 0 if and only if f is the zero vector (that is, the zero function) in C0,.
Definition If V is a vector space with an inner product ,, then we define the norm of any vector v Vtobe v v,v . If u 1, then we say that u is a unit vector.
As before, if we are given a vector v V, then a unit vector in the same “direction” as v is u 1 v. v
Example Let V be the function space C0, with inner product as defined above. Consider the following functions which are all members of C0,: f0t 1
f1t cost
g1t sint
f2t cos2t
g2t sin2t
f3t cos3t Find the norms of these functions and find a “unit function” corresponding to each of them. (In other words, for each of the above functions, fi, find a function i, such that i 1 and i is a scalar multiple of fi, and for each of the functions, gi, find a function i, such that i 1 and i is a scalar multiple of gi.) Solution 2 f0 f0,f0 2 f0x dx 1dx 2
19 shows that
f0 2 .
Therefore, a unit function in the same direction as f0 is 1 0 f0. 2 This is the constant function defined by 1 0t for all t ,. 2
Next, we find 1: 2 f1 f1,f1 2 f1x dx cos2xdx shows that
f1 .
Therefore, a unit function in the same direction as f1 is 1 1 f1. This is the function defined by 1 1t cost for all t ,.
Next, we find 1: 2 g1 g1,g1 2 g1x dx sin2xdx shows that
g1 .
Therefore, a unit function in the same direction as g1 is 1 1 g1. This is the function defined by 1 1t sint for all t ,.
Next, we find 2:
20 2 f2 f2,f2 2 f2x dx cos22xdx shows that
f2 .
Therefore, a unit function in the same direction as f2 is 1 2 f2. This is the function defined by 1 2t cos2t for all t ,.
Does there seem to be a pattern here? There is. In fact, it can be shown that 1 0t 2 1 nt cosnt, n 1 1 nt sinnt, n 1.
Definition If V is a vector space with an inner product , and B is a basis for V, then B is called an orthonormal basis if every vector in B is a unit vector and any two vectors in B are orthogonal to each other.
n Example The standard basis E e1,e2,,en is an orthonormal basis for .
Example The basis 3 4 B 5 , 5 4 3 5 5 is an orthonormal basis for 2.
21 Theorem If V is a vector space with an inner product , and with orthonormal basis B b1,b2,,bn , and if v is any vector in V, then
v,b1
v,b2 vB .
v,bn In other words,
v v,b1 b1 v,b2 b2 v,bn bn. Proof Since B is a basis for V, we know that v can be written as
v c1b1 c2b2 cnbn. Note that
v,b1 b1,v
b1,c1b1 c2b2 cnbn
b1,c1b1 b1,c2b2 b1,cnbn
c1b1,b1 c2b1,b2 cnb1,bn 2 c1b1
c1 Likewise, it can be shown that c2 v,b2 ,,cn v,bn .
Example The basis 3 4 5 5 B b1,b2 , 4 3 5 5 is an orthonormal basis for 2. Find the coordinates of the vector 10 v 5 relative to this basis. Solution 3 4 v,b1 10 5 2 5 5 and 4 3 v,b2 10 5 11. 5 5 Thus,
v 2b1 11b2.
22 The most interesting and powerful applications of these ideas come into play when we are dealing with function spaces, which are infinite–dimensional vector spaces. We cannot study this in great detail in this course. However, we can get a feel for the basic ideas. Here is our main example: As we saw in an earlier example, every one of the functions 1 0t 2 1 nt cosnt, n 1 1 nt sinnt, n 1. has norm 1. It is also true that any two functions in this set are orthogonal to each other. For example, 1,3 1t3tdt 1 cost 1 sin3t dt 0 (because the integrand is an odd function), and 1,3 1t3tdt 1 cost 1 cos3t dt 1 costcos3tdt 0 (is seen by using integration by parts).
Thus, the set of functions 0,1,1,2,2,3, is an orthonormal set of functions in the function space C0,. Motivated by the previous theorem (which pertains to finite–dimensional vector spaces), we are led to ask the following question: Given any function f C0,,is it true that
f f,0 0 f,1 1 f,1 1 f,2 2 f,2 2? In other words, is it true that f,0 f,1 f,1 f,2 f,2 ft cost sint cos2t sin2t 2 for all t ,? Under certain (rather small) restrictions on the function f, it can be proved that the answer to this question is “Yes”. We cannot delve into all of the details, which would require a course in advanced calculus at the minimum. Let us just point out that the
23 infinite trigonometric series on the right is called the Fourier series of the function f and that it is often true that f is in fact equal to its Fourier series. The coefficients f,0 f,1 f,1 f,2 f,2 , , , , , 2 that appear in the Fourier Series are called the Fourier coefficients of f.
Example The function fx x2 is a function in C0,. Find the first ten Fourier coefficients of f and compare the graphs of f with the graphs of the “truncated” Fourier series of f consisting of the first ten terms of the series. Solution
f,0 2 1 x2 1 dx 2 2 2 3 f,1 1 x2 1 cosx dx 4 f,1 1 x2 1 sinx dx 0 f,2 1 x2 1 cos2x dx 1
f,2 0 f,3 1 x2 1 cos3x dx 4 9
f,3 0 f,4 1 x2 1 cos4x dx 1 4
f,4 0 f,5 1 x2 1 cos5x dx 4 . 25 A truncated Fourier series for fx x2 is thus 2 4cos x cos 2x 4 cos 3x 1 cos 4x 4 cos 5x . 3 9 4 25 The graph of this function and the graph of fx x2 are shown below.
24 24 22 20 18 16 14 12 10 8 6 4 2 0 -4 -2 2x 4
25