CHAPTER 3 Euclidean Space n We define the geometric concepts of length, distance, angle and perpendicularity for R . This gives n R the structure of an Euclidean Space. 3.1 Inner Product n We write a point u 2 R as a column vector, i.e. 1 × n matrix. n Definition 3.1. The inner product of u; v 2 R , i.e. 0 1 0 1 u1 v1 B . C B . C u = @ . A ; v = @ . A un vn is given by 0 1 v1 n T B . C X u · v := u v = u1 ··· un @ . A = uivi 2 R i=1 vn Note. To avoid confusion, I will omit the dot for scalar multiplication: I use cu instead of c · u. Some easily checked properties: -17- Chapter 3. Euclidean Space 3.1. Inner Product n Theorem 3.2. Let u; v; w 2 R ; c 2 R (1) u · v = v · u (2)( u + v) · w = u · w + v · w (3)( cu) · v = c(u · v) = u · (cv) (4) u · u ≥ 0 and u · u = 0 iff u = 0. More generally: Definition 3.3. Any vector space V over R equipped with an inner product V ×V −! R satisfying n Theorem 3.2 is called an inner product space. When V = R it is called an Euclidean space. Example 3.1 (Optional). An example of inner product space that is infinite dimensional: Let C[a; b] be the vector space of real-valued continuous function defined on a closed interval [a; b] ⊂ R. Then for f; g 2 C[a; b], Z b f · g := f(t)g(t)dt a gives an inner product on C[a; b]. Remark. All the Definitions and Theorems below applies to inner product spaces. Remark. When K = C, the inner product involves the complex conjugate 0 1 v1 n ∗ B . C X u · v := u v := u1 ··· un @ . A = uivi 2 C i=1 vn so that the last property (4) can hold. Also the third property have to be replaced by (3*)( cu) · v = c(u · v) = u · (cv) Properties 3.4. If A = (aij) is an m × n matrix, then the matrix entries are given by 0 aij = ei · Aej n 0 m where fejg is the standard basis for R and feig is the standard basis for R . -18- Chapter 3. Euclidean Space 3.1. Inner Product Definition 3.5. The norm (or length) of v is the nonnegative scalar p q 2 2 kvk := v · v = v1 + ··· + vn 2 R≥0 For c 2 R, we have kcvk = jcjkvk. Definition 3.6. The vector u with unit length, i.e. kuk = 1 is called a unit vector. 1 Given v 6= 0, kvk v has unit length and is called the normalization of v 0 1 1 p B −2 C 4 p Example 3.2.v = B C 2 R has norm kvk = 12 + (−2)2 + 22 + 02 = 9 = 3. @ 2 A 0 0 1=3 1 1 B −2=3 C v = B C 3 @ 2=3 A 0 is a unit vector. n Definition 3.7. The distance between u; v 2 R is defined by dist(u; v) := ku − vk Theorem 3.8 (Law of cosine). The angle θ between u and v can be calculated by u · v = kukkvk cos θ When θ = 90◦, we have -19- Chapter 3. Euclidean Space 3.2. Orthogonal Basis n Definition 3.9. Two vectors u; v 2 R are orthogonal (or perpendicular) to each other if u · v = 0. Properties 3.10 (Pythagorean Theorem). If u · v = 0, ku + vk2 = kuk2 + kvk2 n Properties 3.11 (Cauchy-Schwarz Inequality). For all u; v 2 R , ju · vj ≤ kukkvk n Properties 3.12 (Triangle Inequality). For all u; v 2 R , ku + vk ≤ kuk + kvk 1 1 Example 3.3. and are orthogonal to each other in 2. 1 −1 R n Example 3.4.0 is orthogonal to every vector in R . 3.2 Orthogonal Basis n Definition 3.13. Let S = fu1; :::; upg 2 R . • S is called an orthogonal set if ui · uj = 0 for all i 6= j. n • If in addition S is a basis of W ⊂ R , it is called an orthogonal basis for W . • If in addition all vectors in S has unit norm, it is called an orthonormal basis for W . n Example 3.5. The standard basis fe1; e2; ··· eng for R is an orthonormal basis. 1 1 Example 3.6. The set ; is an orthogonal basis for 2. 1 −1 R ( p1 ! p1 !) Its rescaled version, the set 2 ; 2 is an orthonormal basis for 2. p1 − p1 R 2 2 -20- Chapter 3. Euclidean Space 3.3. Orthogonal Projection n Theorem 3.14. Let B = fu1; :::; upg be an orthogonal basis for a subspace W ⊂ R . Then for x 2 W we can solve for the coordinates with respect to B explicitly as 0 1 c1 B . C [x]B = @ . A cn where x · ui ci = ; i = 1; :::; p ui · ui 3.3 Orthogonal Projection n Definition 3.15. Let W ⊂ R be a subspace. The orthogonal complement of W is the set ? n W := fv 2 R : v · w = 0 for every w 2 W g Properties 3.16. We have the following properties: ? n • W is a subspace of R . • If L = W ?, then W = L?. • x 2 W ? iff x is orthogonal to every vector in a spanning set ot W . Theorem 3.17. Let A 2 Mm×n(R). Then (RowA)? = NulA; (ColA)? = NulAT Definition 3.18. The orthogonal projection of b onto u is given by b · u Proj (b) := (b · e)e = u u u · u u where e := kuk is the normalization of u. -21- Chapter 3. Euclidean Space 3.3. Orthogonal Projection n n Theorem 3.19 (Orthogonal Decomposition Theorem). Let W ⊂ R a subspace. Then each x 2 R can be written uniquely in the form x = xb + z ? where xb 2 W and z 2 W . Therefore we have n ? R = W ⊕ W n n We sometimes write ProjW (x) := xb. Note that ProjW 2 L(R ; R ) with ? Im(ProjW ) = W; Ker(ProjW ) = W Proof. Explicitly, if fu1; :::; upg is an orthogonal basis of W , then x = Proj (x) + ··· + Proj (x) b u1 up x · u1 x · up = u1 + ··· + up u1 · u1 up · up and z = x − xb. W ? z x Proj (x) u2 u2 u1 Proj (x) u1 W = Span(u1; u2) ProjW (x) Figure 3.1: Orthogonal Projection Remark. In particular, the uniqueness statement says that the orthogonal decomposition, i.e. the formula for xb, does not depend on the basis used for W in the proof. -22- Chapter 3. Euclidean Space 3.3. Orthogonal Projection Properties 3.20. If x 2 W , then ProjW (x) = x. By using orthonormal basis, we can represent ProjW as a matrix: n Theorem 3.21. If fu1; :::; upg is an orthonormal basis for W ⊂ R , then ProjW (x) = (x · u1)u1 + ··· (x · up)up Equivalently, if U = u1 ··· up is an n × p matrix, then T ProjW (x) = UU x The matrix P := UU T is an n × n matrix which is called an orthogonal projection matrix. Definition 3.22. A projection matrix is an n × n matrix such that P 2 = P It is an orthogonal projection matrix if in addition P T = P 011 0 1 1 Example 3.7. If W = Span(v1; v2) where v1 = @0A ; v2 = @ 1 A then fv1; v2g is an orthogonal 1 −1 basis for W : v1 · v2 = 0. The normalization 1 0 p1 1 0 p 1 v 2 v 3 1 B C 2 B p1 C = 0 ; = 3 kv1k @ 1 A kv2k @ A p − p1 2 3 is then an orthonormal basis for W . We have 0 p1 p1 1 2 3 U = B 0 p1 C @ 3 A p1 − p1 2 3 and therefore 0 1 1 1 p p ! 0 5 1 1 1 2 3 p1 0 p1 6 3 6 T B 0 p1 C 2 2 1 1 1 ProjW = UU = 3 1 1 1 = @ 3 3 3 A @ A p p − p 1 1 5 p1 − p1 3 3 3 − 2 3 6 3 6 -23- Chapter 3. Euclidean Space 3.4. Orthogonal Matrix n n Theorem 3.23 (Best Approximation Theorem). Let W be a subspace of R and x 2 R . Then kx − ProjW xk ≤ kx − vk; for any v 2 W i.e. ProjW x 2 W is the closest point in W to x. 3.4 Orthogonal Matrix Definition 3.24. A linear transformation T 2 L(V; W ) between inner product spaces is called an isometry if it preserves the inner product: (T u) · (T v) = u · v for any vector u; v 2 V . n m Theorem 3.25. If T 2 L(R ; R ) is a linear isometry which is represented by an m × n matrix n U, then for x; y 2 R : • U has orthonormal columns T • U U = Idn×n •k Uxk = kxk (i.e. it preserves length) • (Ux) · (Uy) = 0 iff x · y = 0 (i.e. it preserves right angle) Definition 3.26. If n = m, the square matrix U corresponding to a linear isometry is called an orthogonal matrix. It is invertible with U−1 = UT The set of n × n orthogonal matrices is denoted by O(n). -24- Chapter 3. Euclidean Space 3.5. Gram-Schmidt Process Properties 3.27. Orthogonal matrices satisfy the following \group properties": • Idn×n 2 O(n).