Orthogonal Complements (Revised Version) Math 108A: May 19, 2010 John Douglas Moore 1 The dot product You will recall that the dot product was discussed in earlier calculus courses. If n x = (x1: : : : ; xn) and y = (y1: : : : ; yn) are elements of R , we define their dot product by x · y = x1y1 + ··· + xnyn: The dot product satisfies several key axioms: 1. it is symmetric: x · y = y · x; 2. it is bilinear: (ax + x0) · y = a(x · y) + x0 · y; 3. and it is positive-definite: x · x ≥ 0 and x · x = 0 if and only if x = 0. The dot product is an example of an inner product on the vector space V = Rn over R; inner products will be treated thoroughly in Chapter 6 of [1]. Recall that the length of an element x 2 Rn is defined by p jxj = x · x: Note that the length of an element x 2 Rn is always nonnegative. Cauchy-Schwarz Theorem. If x 6= 0 and y 6= 0, then x · y −1 ≤ ≤ 1: (1) jxjjyj Sketch of proof: If v is any element of Rn, then v · v ≥ 0. Hence (x(y · y) − y(x · y)) · (x(y · y) − y(x · y)) ≥ 0: Expanding using the axioms for dot product yields (x · x)(y · y)2 − 2(x · y)2(y · y) + (x · y)2(y · y) ≥ 0 or (x · x)(y · y)2 ≥ (x · y)2(y · y): 1 Dividing by y · y, we obtain (x · y)2 jxj2jyj2 ≥ (x · y)2 or ≤ 1; jxj2jyj2 and (1) follows by taking the square root. The key point of the Cauchy-Schwarz Inequality (1) is that it allows us to define angles between vectors x and y in Rn. (It is a first step towards extending geometry from R2 and R3 to Rn.) It follows from properties of the cosine function that given a number t 2 [−1; 1], there is a unique angle θ such that θ 2 [0; π] and cos θ = t: Thus we can define the angle between two nonzero vectors x and y in Rn by requiring that x · y θ 2 [0; π] and cos θ = : jxjjyj Then the dot product satisfies the formula x · y = jxjjyj cos θ: In particular, we can say that two vectors vectors x and y in Rn are perpen- dicular or orthogonal if x · y = 0. This provides much intuition for dealing with vectors in Rn. n Thus if a = (a1; : : : an) is a nonzero element of R , the homogeneous linear equation a1x1 + ··· + anxn = 0 n describes the set of all vectors x = (x1; : : : ; xn) 2 R that are perpendicular to a. The set of solutions n W = fx 2 R : a · x = 0g to this homogeneous linear equation is a linear subspace of Rn. We remind you that to see that W is a linear subspace, we need to check three facts: 1. a · 0 = 0, so 0 2 W . 2. If x 2 W and y 2 W , then a · x = 0 and a · y = 0, and it follows from the axioms for dot product that a · (x + y) = 0 so x + y 2 W . 3. If c 2 R and x 2 W , then a · x = 0, and it follows from the axioms for dot product that a · (cx) = 0, so cx 2 W . One can also show that W is a linear subspace of Rn by showing that it is the null space of a linear map, as described in the following section. (Recall that the null space of a linear map is always a subspace by a theorem in the text.) 2 2 Linear systems and orthogonal complements Recall that linear algebra is the theory behind solving systems of linear equa- tions, such as a11x1 + a12x2 + ··· + a1nxn = b1; a x + a x + ··· + a x = b ; 21 1 22 2 2n n 2 (2) ········ am1x1 + am2x2 + ··· + amnxn = bm: Here the aij's and bi's are known elements of the field F, and we are solving for the unknowns x1; : : : ; xn. This system of linear equations can be written in terms of the matrix 0 1 a11 a12 ··· a1n B a21 a22 ··· a2n C A = B C : @ ······ A am1 am2 ··· amn as Ax = b, where 0 1 0 1 x1 b1 Bx2 C B b2 C x = B C and b = B C : @ · A @ · A xn bm The matrix A defines a linear map n m LA : F ! F by LA(x) = Ax: The range R(LA) of LA is just the space of vectors b for which the equation Ax = b has a solution, while the null space N(LA) of LA is the space of solutions to the associated homogeneous linear system a11x1 + a12x2 + ··· + a1nxn = 0; a x + a x + ··· + a x = 0; 21 1 22 2 2n n (3) ········ am1x1 + am2x2 + ··· + amnxn = 0: The fundamental theorem of Chapter 2 from the text [1] states that n n = dim(F ) = dim(N(LA)) + dim(R(LA)): (4) In particular, it follows that dim(N(LA)) = 0 , dim(R(TA)) = n: (5) In the special case where n = m, (5) implies that LA is surjective if and only if it is injective. Equivalently, the system (2) has a solution for every choice of vector b 2 Rn if and only if the associated homogeneous system (3) has only the zero solution. 3 In the special case where F = R, the notion of orthogonal complement gives a nice picture of the process of finding a basis for the space of solutions to a homogeneous linear system. We can use the dot product to write the homogeneous linear system whose solutions form the null space of LA in a particularly intuitive form. Let a1 = (a11; a12; : : : ; a1n); a2 = (a21; a12; : : : ; a2n); ····· am = (am1; am2; : : : ; amn); and let W = span(a1; a2;:::; am). We can then say that n N(LA) = fx 2 R : a1 · x = 0; a2 · x = 0; ··· ; am · x = 0g: Note that the last space on the right-hand side is simply the collection of all vectors which are perpendicular to W . We call this space the orthogonal com- plement to W and denote it by W ?. We can then write ? n N(LA) = W = fx 2 R : a · x = 0 for a 2 W g: Definition 1. Suppose that W1 and W2 are linear subspaces of a vector space V . We let W1 + W2 = fv 2 V : v = w1 + w2, where w1 2 W1 and w2 2 W2 g: One readily proves that W1 + W2 is a linear subspace of V . Definition 2. Suppose that W1 and W2 are linear subspaces of a vector space V . We say that V is the direct sum of W1 and W2, and write V = W1 ⊕ W2, if 1. W1 \ W2 = f0g, and 2. V = W1 + W2, that is, if v 2 V , then v = w1 + w2, where w1 2 W1 and w2 2 W2. Proposition 1. If V = W1 ⊕ W2, then dim V = dim W1 + dim W2: Proof: It suffices to show that if (u1;:::; um) is a basis for W1 and (w1;:::; wn) is a basis for W2, then (u1;:::; um; w1;:::; wn) (6) is a basis for V . So we need to show that (6) is linearly independent and spans V . To show that (6) is linearly independent, we suppose that a1u1 + · + amum + b1w1 + ··· + bnwn = 0: 4 Then a1u1 + · + amum = −b1w1 − · · · − bnwn 2 W1 \ W2 = f0g: But then a1 = ··· = am = 0 because (u1;:::; um) is a basis for W1, while b1 = ··· = bn = 0 because (w1;:::; wn) is a basis for W2. This implies that (6) is linearly independent. To see that (6) spans V , we suppose that v 2 V . Then V = W1 + W2 ) v = w1 + w2; where w1 2 W1 and w2 2 W2. Since (u1;:::; um) is a basis for W1, w1 = a1u1 + · + amum; for some a1; : : : am 2 F. Since (w1;:::; wn) is a basis for W2 is a basis for W2, w2 = b1w1 + ··· + bnwn; for some b1; : : : bn 2 F. Hence v = a1u1 + · + amum + b1w1 + ··· + bnwn; and (6) spans V . QED n Proposition 2. Suppose that W1 and W2 are linear subspaces of R such that 1. W1 \ W2 = f0g, and 2. dim W1 + dim W2 ≥ V , then V = W1 ⊕ W2. Proof: Suppose that (u1;:::; um) is a basis for W1 and (w1;:::; wn) is a basis for W2. Then by the first part of the argument for Proposition 1, γ = (u1;:::; um; w1;:::; wn) is linearly independent within V . We know that γ can be extended to a basis for V , but the second condition implies that this basis must have ≤ m+n elements. Therefore, the extension must be γ itself, and hence V = W1 + W2. Thus W1 and W2 satisfy both conditions for direct sum. QED Orthogonal Complement Theorem. If (a1; a2;:::; am) is a list of vectors n ? n in R , W = span(a1; a2;:::; am) and W is the space of solutions x 2 R to the homogeneous linear system of equations a1 · x = 0; a2 · x = 0; ··· ; am · x = 0; 5 then Rn = W ⊕ W ?. Proof: First note that if x 2 W \ W ?, then x · x = 0, and hence jxj = 0. This can only happen if x = 0, so W \ W ? = f0g: The list (a1; a2;:::; am) can be reduced to a basis (b1; b2;:::; bk) for W , where k = dim W . Let B be the k × n matrix whose rows are the elements of n k the list (b1; b2;:::; bk) and let LB : R ! R be the corresponding linear map. ? Then we can write W = null(LB), and it follows from (4) that n n = dim(R ) = dim(null(LB)) + dim(range(LB)) ≤ dim W ? + k = dim W ? + dim W: It now follows from Proposition 2 that Rn = W ⊕ W ?, finishing the proof of the Theorem.