Orthogonal Complements (Revised Version)

Orthogonal Complements (Revised Version)

Orthogonal Complements (Revised Version) Math 108A: May 19, 2010 John Douglas Moore 1 The dot product You will recall that the dot product was discussed in earlier calculus courses. If n x = (x1: : : : ; xn) and y = (y1: : : : ; yn) are elements of R , we define their dot product by x · y = x1y1 + ··· + xnyn: The dot product satisfies several key axioms: 1. it is symmetric: x · y = y · x; 2. it is bilinear: (ax + x0) · y = a(x · y) + x0 · y; 3. and it is positive-definite: x · x ≥ 0 and x · x = 0 if and only if x = 0. The dot product is an example of an inner product on the vector space V = Rn over R; inner products will be treated thoroughly in Chapter 6 of [1]. Recall that the length of an element x 2 Rn is defined by p jxj = x · x: Note that the length of an element x 2 Rn is always nonnegative. Cauchy-Schwarz Theorem. If x 6= 0 and y 6= 0, then x · y −1 ≤ ≤ 1: (1) jxjjyj Sketch of proof: If v is any element of Rn, then v · v ≥ 0. Hence (x(y · y) − y(x · y)) · (x(y · y) − y(x · y)) ≥ 0: Expanding using the axioms for dot product yields (x · x)(y · y)2 − 2(x · y)2(y · y) + (x · y)2(y · y) ≥ 0 or (x · x)(y · y)2 ≥ (x · y)2(y · y): 1 Dividing by y · y, we obtain (x · y)2 jxj2jyj2 ≥ (x · y)2 or ≤ 1; jxj2jyj2 and (1) follows by taking the square root. The key point of the Cauchy-Schwarz Inequality (1) is that it allows us to define angles between vectors x and y in Rn. (It is a first step towards extending geometry from R2 and R3 to Rn.) It follows from properties of the cosine function that given a number t 2 [−1; 1], there is a unique angle θ such that θ 2 [0; π] and cos θ = t: Thus we can define the angle between two nonzero vectors x and y in Rn by requiring that x · y θ 2 [0; π] and cos θ = : jxjjyj Then the dot product satisfies the formula x · y = jxjjyj cos θ: In particular, we can say that two vectors vectors x and y in Rn are perpen- dicular or orthogonal if x · y = 0. This provides much intuition for dealing with vectors in Rn. n Thus if a = (a1; : : : an) is a nonzero element of R , the homogeneous linear equation a1x1 + ··· + anxn = 0 n describes the set of all vectors x = (x1; : : : ; xn) 2 R that are perpendicular to a. The set of solutions n W = fx 2 R : a · x = 0g to this homogeneous linear equation is a linear subspace of Rn. We remind you that to see that W is a linear subspace, we need to check three facts: 1. a · 0 = 0, so 0 2 W . 2. If x 2 W and y 2 W , then a · x = 0 and a · y = 0, and it follows from the axioms for dot product that a · (x + y) = 0 so x + y 2 W . 3. If c 2 R and x 2 W , then a · x = 0, and it follows from the axioms for dot product that a · (cx) = 0, so cx 2 W . One can also show that W is a linear subspace of Rn by showing that it is the null space of a linear map, as described in the following section. (Recall that the null space of a linear map is always a subspace by a theorem in the text.) 2 2 Linear systems and orthogonal complements Recall that linear algebra is the theory behind solving systems of linear equa- tions, such as a11x1 + a12x2 + ··· + a1nxn = b1; a x + a x + ··· + a x = b ; 21 1 22 2 2n n 2 (2) ········ am1x1 + am2x2 + ··· + amnxn = bm: Here the aij's and bi's are known elements of the field F, and we are solving for the unknowns x1; : : : ; xn. This system of linear equations can be written in terms of the matrix 0 1 a11 a12 ··· a1n B a21 a22 ··· a2n C A = B C : @ ······ A am1 am2 ··· amn as Ax = b, where 0 1 0 1 x1 b1 Bx2 C B b2 C x = B C and b = B C : @ · A @ · A xn bm The matrix A defines a linear map n m LA : F ! F by LA(x) = Ax: The range R(LA) of LA is just the space of vectors b for which the equation Ax = b has a solution, while the null space N(LA) of LA is the space of solutions to the associated homogeneous linear system a11x1 + a12x2 + ··· + a1nxn = 0; a x + a x + ··· + a x = 0; 21 1 22 2 2n n (3) ········ am1x1 + am2x2 + ··· + amnxn = 0: The fundamental theorem of Chapter 2 from the text [1] states that n n = dim(F ) = dim(N(LA)) + dim(R(LA)): (4) In particular, it follows that dim(N(LA)) = 0 , dim(R(TA)) = n: (5) In the special case where n = m, (5) implies that LA is surjective if and only if it is injective. Equivalently, the system (2) has a solution for every choice of vector b 2 Rn if and only if the associated homogeneous system (3) has only the zero solution. 3 In the special case where F = R, the notion of orthogonal complement gives a nice picture of the process of finding a basis for the space of solutions to a homogeneous linear system. We can use the dot product to write the homogeneous linear system whose solutions form the null space of LA in a particularly intuitive form. Let a1 = (a11; a12; : : : ; a1n); a2 = (a21; a12; : : : ; a2n); ····· am = (am1; am2; : : : ; amn); and let W = span(a1; a2;:::; am). We can then say that n N(LA) = fx 2 R : a1 · x = 0; a2 · x = 0; ··· ; am · x = 0g: Note that the last space on the right-hand side is simply the collection of all vectors which are perpendicular to W . We call this space the orthogonal com- plement to W and denote it by W ?. We can then write ? n N(LA) = W = fx 2 R : a · x = 0 for a 2 W g: Definition 1. Suppose that W1 and W2 are linear subspaces of a vector space V . We let W1 + W2 = fv 2 V : v = w1 + w2, where w1 2 W1 and w2 2 W2 g: One readily proves that W1 + W2 is a linear subspace of V . Definition 2. Suppose that W1 and W2 are linear subspaces of a vector space V . We say that V is the direct sum of W1 and W2, and write V = W1 ⊕ W2, if 1. W1 \ W2 = f0g, and 2. V = W1 + W2, that is, if v 2 V , then v = w1 + w2, where w1 2 W1 and w2 2 W2. Proposition 1. If V = W1 ⊕ W2, then dim V = dim W1 + dim W2: Proof: It suffices to show that if (u1;:::; um) is a basis for W1 and (w1;:::; wn) is a basis for W2, then (u1;:::; um; w1;:::; wn) (6) is a basis for V . So we need to show that (6) is linearly independent and spans V . To show that (6) is linearly independent, we suppose that a1u1 + · + amum + b1w1 + ··· + bnwn = 0: 4 Then a1u1 + · + amum = −b1w1 − · · · − bnwn 2 W1 \ W2 = f0g: But then a1 = ··· = am = 0 because (u1;:::; um) is a basis for W1, while b1 = ··· = bn = 0 because (w1;:::; wn) is a basis for W2. This implies that (6) is linearly independent. To see that (6) spans V , we suppose that v 2 V . Then V = W1 + W2 ) v = w1 + w2; where w1 2 W1 and w2 2 W2. Since (u1;:::; um) is a basis for W1, w1 = a1u1 + · + amum; for some a1; : : : am 2 F. Since (w1;:::; wn) is a basis for W2 is a basis for W2, w2 = b1w1 + ··· + bnwn; for some b1; : : : bn 2 F. Hence v = a1u1 + · + amum + b1w1 + ··· + bnwn; and (6) spans V . QED n Proposition 2. Suppose that W1 and W2 are linear subspaces of R such that 1. W1 \ W2 = f0g, and 2. dim W1 + dim W2 ≥ V , then V = W1 ⊕ W2. Proof: Suppose that (u1;:::; um) is a basis for W1 and (w1;:::; wn) is a basis for W2. Then by the first part of the argument for Proposition 1, γ = (u1;:::; um; w1;:::; wn) is linearly independent within V . We know that γ can be extended to a basis for V , but the second condition implies that this basis must have ≤ m+n elements. Therefore, the extension must be γ itself, and hence V = W1 + W2. Thus W1 and W2 satisfy both conditions for direct sum. QED Orthogonal Complement Theorem. If (a1; a2;:::; am) is a list of vectors n ? n in R , W = span(a1; a2;:::; am) and W is the space of solutions x 2 R to the homogeneous linear system of equations a1 · x = 0; a2 · x = 0; ··· ; am · x = 0; 5 then Rn = W ⊕ W ?. Proof: First note that if x 2 W \ W ?, then x · x = 0, and hence jxj = 0. This can only happen if x = 0, so W \ W ? = f0g: The list (a1; a2;:::; am) can be reduced to a basis (b1; b2;:::; bk) for W , where k = dim W . Let B be the k × n matrix whose rows are the elements of n k the list (b1; b2;:::; bk) and let LB : R ! R be the corresponding linear map. ? Then we can write W = null(LB), and it follows from (4) that n n = dim(R ) = dim(null(LB)) + dim(range(LB)) ≤ dim W ? + k = dim W ? + dim W: It now follows from Proposition 2 that Rn = W ⊕ W ?, finishing the proof of the Theorem.

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    8 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us