The transpose Dot product

T Definition. Let A be an m n matrix whose Definition. Let u =[u1, , un] and × T T ··· n (i, j)-th entry is aij.ThendefineA to be the v =[v1, , vn] be vectors in R .Thendefinethe n m matrix whose (i, j)-th entry is aji.Wesay dot product··· u v of u and v to be the number × AT is the transpose of A.Inotherwords,AT is · T obtained from A by interchanging the rows into u v = u v = u1v1 + u2v2 + + unvn. columns. · ··· Properties. The following properties are easily Rn Properties: verified from the definition. Let u, v, w and c R. ∈ ∈ (AT )T = A. ◦ u v = v u. ◦ · · ( + )T = T + T A B A B . (u + w) v = u v + w v. ◦ ◦ · · · T T (cA) = cA . (cu) v = c(u v). ◦ ◦ · · (AB)T = BT AT . u u ! 0 for all u and u u = 0 if and only if ◦ ◦ u ·= 0. · det(AT )=det(A). ◦ Definition. Length of a vector v Rn is defined to Example. be ∈ v = √v v = v2 + v2 + + v2 . ∥ ∥ · 1 2 ··· n By Pythagorus theorem,! this matches with our usual definition of length for R2 and R3. Distances and angles The dot product gives us a convenient way to Using the propeties of the dot product we can work with distances and angles between vectors calculate u + v 2 in another way: in Rn. ∥ ∥ u + v 2 =(u + v) (u + v) Rn Let u, v .Thenthedistancebetweenu and v ∥ ∥ = u u + u· v + v u + v v is given∈ by · · · · = u 2 + v 2 + 2u v. ∥ ∥ ∥ ∥ · dist(u, v)= u − v = (u − v) (u − v). ∥ ∥ · Equating the two expressions for u + v 2,we ∥ ∥ For example, the distance between" u =[7, −9, 4]T find: T and v =[6, −7, 2] is u 2 + v 2 + 2 u v cos θ = u 2 + v 2 + 2u v, ∥ ∥ ∥ ∥ ∥ ∥∥ ∥ ∥ ∥ ∥ ∥ · u − v = [1, −2, 2]T = √1 + 4 + 4 = 3. so ∥ ∥ ∥ ∥ u v cos θ = u v. ∥ ∥∥ ∥ · This leads us to define the angle between two n Next, let us think about angles. Let u, v R3 (or nonzero vectors u and v in R to be ∈ even R2). Let θ be the angle between u and v u v angle(u, v)=cos−1 · . (going anticlockwise from u to v). From the u v parallelogram law in geometry, we have, #∥ ∥∥ ∥$ For example, verify that 2 2 2 u + v = u + v + 2 u v cos θ. ([ − ]T [ − ]T)= −1(− )= ∥ ∥ ∥ ∥ ∥ ∥ ∥ ∥∥ ∥ angle 1, 1, 0 , 0, 1, 1 cos 1/2 π/3. Orthogonality,Orthogonal complements Definition. Let u, v Rn.Defineu and v to be Properties Let S be a nonempty subset of Rn. orthogonal,ifu v =∈ 0. n · S⊥ is a subspace of R . If u, v are nonzero and the angle between them is ◦ θ,then (span(S))⊥ = S⊥. u v = u v cos θ. ◦ · ∥ ∥∥ ∥ If S is a nonempty subspace of Rn,then = So u and v are orthogonal means θ π/2, that is ◦ (S ) = S. the vectors are perpendicular. For example, verify ⊥ ⊥ that The proofs are left as exercises. [3, −5, 2]T [−1, 3, 9]T = −3 − 15 + 18 = 0, · Observation. Let A be an n k matrix. Let S be the set of columns of A.Thenobservethat× so these two vectors are orthogonal. T S⊥ = nul(A ). Definition. Let S be a nonempty subset of Rn. Rn Define S⊥ to be the set of all vectors in that are This allows us to compute the basis for S⊥,given n orthogonal to every vector in S.WesayS⊥ is the any finite set S of vectors in R . orthogonal complement of S.Inpartiular,ifv is n nonzero vector in R ,thenv⊥ is the set of all vectors orthogonal to v. Examples. Orthogonal projection Definition. Fix a nonzero vector u in Rn.LetL be Exercise. Show thaty ˆ is the point on L that is the line in Rn spanned by u. closest to y. Pick y Rn.Letc be a scalar. Note that (y − cu) is orthogonal∈ to u if and only if 0 =(y − cu) u = y u − c(u u) · · · or equivalently y u c = · . u u · Define y u Example. yˆ = proj (y)= · u. L u u · Then y decomposes as y = yˆ +(y − yˆ ) where the first termy ˆ is a multiple of u and the second term (y − yˆ ) is orthogonal to u.Wesayˆy the orthogonal projection of y on L. Orthogonal sets, orthogonal basis Orthonormal Basis Definition. Asetofvectors{u , , u } in Rn is Definition. Let W be a subspace of Rn.AsubsetS 1 ··· p called orthogonal if ui uj = 0 for all i = j. of W is called an orthogonal basis of W if S is a · ̸ basis of W and S is an orthogonal set. Example. Remark. The theorem above impies that a set of k nonzero orthogonal vectors in a k dimensional subspace W always forms an orthogonal basis of W. Theorem. Orthogonal sets of nonzero vectors are Theorem. Let {u1, , up} be an orthogonal basis linearly independent. So they form a basis for the for a subspace W of···Rn.Letw W.Write subspace they span. ∈ w = c1u1 + + cpup. The proofs of the theorems in this section are nice ··· exercises. Try them. If you get stuck, see the Then c = w u /u u . textbook. j · j j · j In other words, the components cjuj are the orthogonal projections of w in the directions of uj. Orthonormal set Orthogonal matrices Definition. AnonemptysubsetS of Rn is called Let U be a n n matrix. The following are an orthonormal set if S is an orthogonal set and equivalent: × each vector in S has length 1.Anorthonormal basis for a subspace W of Rn means a basis of W The columns of U form an orthonormal ◦ Rn that is also an orthonormal set. basis of . UT U = I. Orthonormal = orthogonal + unit length. ◦ Ux Uy = x y for all x, y Rn. ◦ · · ∈ |Ux| = |x| for all x Rn. Examples. ◦ ∈ Such matrices are called orthogonal matrices. The conditions in the above theorem say that these are exactly the matrices U such that the linear maps LU preserve the distances and angles (hence orthogonality). Note that if U is orthogonal, then det(U)= 1.IfU is orthogonal and det(U)=1, then U is± called a rotation.