DOCSLIB.ORG

10 Orthogonality

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
10 Orthogonality 10 Orthogonality 10.1 Orthogonal subspaces In the plane R2 we think of the coordinate axes as being orthogonal (perpen- dicular) to each other. We can express this in terms of vectors by saying that every vector in one axis is orthogonal to every vector in the other. Let V be an inner product space and let S and T be subsets of V . We say that S and T are orthogonal, written S T , if every vector in S is orthogonal to every vector in T : ⊥ S T s t for all s S, t T ⊥ ⇐⇒ ⊥ ∈ ∈ 3 10.1.1 Example In R let S be the x1x2-plane and let T be the x3-axis. Show that S T . ⊥ T Solution Let s S and let t T . We can write s = [x1, x2, 0] and t = T ∈ ∈ [0, 0, x3] so that 0 T s, t = s t = x1 x2 0 0 =0. h i x 3 Therefore, s t and we conclude that S T . ⊥ ⊥ 3 10.1.2 Example In R let S be the x1x2-plane and let T be the x1x3- plane. Is it true that S T ? ⊥ Solution Although the planes S and T appear to be perpendicular in the in- formal sense, they are not so in the technical sense. For instance, the vector T e1 = [1, 0, 0] is in both S and T , yet 1 T e1, e1 = e1 e1 = 1 0 0 0 =1 =0, h i 0 6 so e e . Viewing the first e in S and the second e in T , we see that S T . 1 6⊥ 1 1 1 6⊥ 1 10 ORTHOGONALITY 2 Orthogonal complement. Let S be a subspace of V . The orthogonal complement of S (in V ), written S⊥, is the set of all vectors in V that are orthogonal to every vector in S: S⊥ = v V v s for all s S . { ∈ | ⊥ ∈ } 3 For instance, in R the orthogonal complement of the x1x2-plane is the x3-axis. On the other hand, the orthogonal complement of the x3-axis is the x1x2-plane. It follows using the inner product axioms that if S is a subspace of V , then so is its orthogonal complement S⊥. The next theorem says that in order to show that a vector is orthogonal to a subspace it is enough to check that it is orthogonal to each vector in a spanning set for the subspace. Theorem. Let b , b ,..., bn be a set vectors in V and let { 1 2 } S = Span b , b ,..., bn . A vector v in V is in S⊥ if and only { 1 2 } if v bi for each i. ⊥ Proof. For simplicity of notation, we prove only the special case when n =2 so that S = Span b , b . If v is in S⊥, then it is orthogonal to every vector in { 1 2} S, so, in particular, v b1 and v b2. Now assume that v V and v b1 and v b . If s S,⊥ then we can⊥ write s = αb + αb for some∈ α , α ⊥ R. ⊥ 2 ∈ 1 2 1 2 ∈ 10 ORTHOGONALITY 3 So using properties of the inner product we get v, s = v, α b + α b h i h 1 1 2 2i = α v, b + α v, b (ii), (iii), and (iv) 1h 1i 2h 2i = α 0+ α 0 v b and v b 1 · 2 · ⊥ 1 ⊥ 2 =0. Therefore, v s. This shows that v is orthogonal to every vector in S so that ⊥ v S⊥. ∈ Theorem. If A is a matrix, then the orthogonal complement of the row space of A is the null space of A: (Row A)⊥ = Null A. Proof. Let A be an m n matrix and let b1, b2,..., bm be the rows of A written × n as columns so that Row A = Span b1, b2,..., bm . Let x be a vector in R . The product Ax is the m 1 matrix{ obtained by taking} the dot products of x with the rows of A: × T T b1 b1 x . Ax = . x = . bT bT x m m T so, in view of the previous theorem, x is in (Row A)⊥ if and only if bi x = 0 for each i and this holds if and only if x is in Null A. 10.1.3 Example Let S be the subspace of R4 spanned by the vectors T T [1, 0, 2, 1] and [0, 1, 3, 2] . Find a basis for S⊥. − − Solution We first note that S = Row A, where 1 0 2 1 A = . 01− 3 2 − According to the theorem, S⊥ = (Row A)⊥ = Null A so we need only find a basis for the null space of A. The augmented matrix we write to solve Ax = 0 is already in reduced row echelon form (RREF): 1 0 2 1 0 0 1− 3 2 0 − so Null A = [2t s, 3t +2s,t,s]T t,s R . We have { − − | ∈ } 2t s 2 1 3t−+2s 3 −2 = t + s , − t −1 0 s 0 1 10 ORTHOGONALITY 4 T T and therefore [2, 3, 1, 0] , [ 1, 2, 0, 1] is a basis for Null A (and hence S⊥). { − − } The next theorem generalizes the notion that the shortest distance from a point to a plane is achieved by dropping a perpendicular. Theorem. Let S be a subspace of V , let b be a vector in V and assume that b = s0 + r with s0 S and r S⊥. For every s S we have ∈ ∈ ∈ dist(b, s ) dist(b, s). 0 ≤ Proof. Let s S. We have ∈ b s 2 = (b s ) + (s s) 2 k − k k − 0 0 − k = b s 2 + s s 2 Pythagorean theorem k − 0k k 0 − k b s 2 ≥k − 0k (note that the Pythagorean theorem applies since b s0 = r S⊥ and s0 s S so that (b s ) (s s)). Therefore, − ∈ − ∈ − 0 ⊥ 0 − dist(b, s )= b s b s = dist(b, s). 0 k − 0k≤k − k 10.2 Least squares Suppose that the matrix equation Ax = b has no solution. In terms of distance, this means that dist(b, Ax) is never zero, no matter what x is. 10 ORTHOGONALITY 5 Instead of leaving the equation unsolved, it is sometimes useful to find an x0 that is as close to being a solution as possible, that is, for which the distance from b to Ax0 is less than or equal to the distance from b to Ax for every other x. This is called a “least squares solution.” A least squares solution can be used, for instance, to find the line that best fits some data points (see Example 10.2.1). Least squares. n Let A be an m n matrix and let b be a vector in R . If x = x0 is a solution to× AT Ax = AT b. then, for every x Rn, ∈ dist(b, Ax ) dist(b, Ax). 0 ≤ Such an x0 is called a least squares solution to the equation Ax = b. n T T Proof. Let x0 R and assume that A Ax0 = A b. Letting s0 = Ax0, we have ∈ AT (b s )= AT (b Ax )= 0, − 0 − 0 so that b s Null AT − 0 ∈ T = (Row A )⊥ (Theorem in Section 10.1) = (Col A)⊥ = S⊥, where S = Ax x Rn (the product Ax can be interpreted as the linear combination{ of the| columns∈ } of A with the entries of x as scalar factors, so S is the set of all linear combinations of the columns of A, which is Col A). This shows that the vector r = b s0 is in S⊥, so that b = s0 + r with s0 S and − n ∈ r S⊥. By the last theorem of Section 10.1, for every x R , we have ∈ ∈ dist(b, Ax ) = dist(b, s ) dist(b, Ax). 0 0 ≤ 10.2.1 Example Use a least squares solution to find a line that best fits the data points (1, 2), (3, 2), and (4, 5). 10 ORTHOGONALITY 6 Solution Here is the graph: If we write the desired line as c + dx = y, then ideally the line would go through all three points giving the system c + d =2 c +3d =2 c +4d =5, which can be written as the matrix equation Ax = b, where 1 1 2 c A = 1 3 , x = , b = 2 . d 1 4 5 The least squares solution is obtained by solving the equation AT Ax = AT b. We have 1 1 1 1 1 3 8 AT A = 1 3 = 1 3 4 8 26 1 4 and 2 1 1 1 9 AT b = 2 = , 1 3 4 28 5 so the matrix equation has corresponding augmented matrix 3 8 9 8 3 8 9 − 8 26 28 3 ∼ 0 14 12 1 2 3 8 9 7 ∼ 0 7 6 8 − 1 21 0 15 21 ∼ 0 7 6 1 7 5 1 0 7 ∼ 0 1 6 7 10 ORTHOGONALITY 7 5 6 5 6 Therefore, c = 7 and d = 7 and the best fitting line is y = 7 + 7 x, which is the line shown in the graph. (We could tell in advance that the matrix equation Ax = b has no solution since the points are not collinear. In general, however, it is not necessary to first check that an equation has no solution before applying the least squares method since, if it has a solution, then that is what the method will produce.) We can use the last example to explain the terminology “least squares.” The method gives a line that minimizes the sum of the squares of the vertical dis- 2 2 2 tances from the points to the lines.
Load more

Recommended publications
  • The Grassmann Manifold
    The Grassmann Manifold 1. For vector spaces V and W denote by L(V; W ) the vector space of linear maps from V to W . Thus L(Rk; Rn) may be identified with the space Rk£n of k £ n matrices. An injective linear map u : Rk ! V is called a k-frame in V . The set k n GFk;n = fu 2 L(R ; R ): rank(u) = kg of k-frames in Rn is called the Stiefel manifold. Note that the special case k = n is the general linear group: k k GLk = fa 2 L(R ; R ) : det(a) 6= 0g: The set of all k-dimensional (vector) subspaces ¸ ½ Rn is called the Grassmann n manifold of k-planes in R and denoted by GRk;n or sometimes GRk;n(R) or n GRk(R ). Let k ¼ : GFk;n ! GRk;n; ¼(u) = u(R ) denote the map which assigns to each k-frame u the subspace u(Rk) it spans. ¡1 For ¸ 2 GRk;n the fiber (preimage) ¼ (¸) consists of those k-frames which form a basis for the subspace ¸, i.e. for any u 2 ¼¡1(¸) we have ¡1 ¼ (¸) = fu ± a : a 2 GLkg: Hence we can (and will) view GRk;n as the orbit space of the group action GFk;n £ GLk ! GFk;n :(u; a) 7! u ± a: The exercises below will prove the following n£k Theorem 2. The Stiefel manifold GFk;n is an open subset of the set R of all n £ k matrices. There is a unique differentiable structure on the Grassmann manifold GRk;n such that the map ¼ is a submersion.
    [Show full text]
  • A New Description of Space and Time Using Clifford Multivectors
    A new description of space and time using Clifford multivectors James M. Chappell† , Nicolangelo Iannella† , Azhar Iqbal† , Mark Chappell‡ , Derek Abbott† †School of Electrical and Electronic Engineering, University of Adelaide, South Australia 5005, Australia ‡Griffith Institute, Griffith University, Queensland 4122, Australia Abstract Following the development of the special theory of relativity in 1905, Minkowski pro- posed a unified space and time structure consisting of three space dimensions and one time dimension, with relativistic effects then being natural consequences of this space- time geometry. In this paper, we illustrate how Clifford’s geometric algebra that utilizes multivectors to represent spacetime, provides an elegant mathematical framework for the study of relativistic phenomena. We show, with several examples, how the application of geometric algebra leads to the correct relativistic description of the physical phenomena being considered. This approach not only provides a compact mathematical representa- tion to tackle such phenomena, but also suggests some novel insights into the nature of time. Keywords: Geometric algebra, Clifford space, Spacetime, Multivectors, Algebraic framework 1. Introduction The physical world, based on early investigations, was deemed to possess three inde- pendent freedoms of translation, referred to as the three dimensions of space. This naive conclusion is also supported by more sophisticated analysis such as the existence of only five regular polyhedra and the inverse square force laws. If we lived in a world with four spatial dimensions, for example, we would be able to construct six regular solids, and in arXiv:1205.5195v2 [math-ph] 11 Oct 2012 five dimensions and above we would find only three [1].
    [Show full text]
  • 2 Hilbert Spaces You Should Have Seen Some Examples Last Semester
    2 Hilbert spaces You should have seen some examples last semester. The simplest (finite-dimensional) ex- C • A complex Hilbert space H is a complete normed space over whose norm is derived from an ample is Cn with its standard inner product. It’s worth recalling from linear algebra that if V is inner product. That is, we assume that there is a sesquilinear form ( , ): H H C, linear in · · × → an n-dimensional (complex) vector space, then from any set of n linearly independent vectors we the first variable and conjugate linear in the second, such that can manufacture an orthonormal basis e1, e2,..., en using the Gram-Schmidt process. In terms of this basis we can write any v V in the form (f ,д) = (д, f ), ∈ v = a e , a = (v, e ) (f , f ) 0 f H, and (f , f ) = 0 = f = 0. i i i i ≥ ∀ ∈ ⇒ ∑ The norm and inner product are related by which can be derived by taking the inner product of the equation v = aiei with ei. We also have n ∑ (f , f ) = f 2. ∥ ∥ v 2 = a 2. ∥ ∥ | i | i=1 We will always assume that H is separable (has a countable dense subset). ∑ Standard infinite-dimensional examples are l2(N) or l2(Z), the space of square-summable As usual for a normed space, the distance on H is given byd(f ,д) = f д = (f д, f д). • • ∥ − ∥ − − sequences, and L2(Ω) where Ω is a measurable subset of Rn. The Cauchy-Schwarz and triangle inequalities, • √ (f ,д) f д , f + д f + д , | | ≤ ∥ ∥∥ ∥ ∥ ∥ ≤ ∥ ∥ ∥ ∥ 2.1 Orthogonality can be derived fairly easily from the inner product.
    [Show full text]
  • Lecture 4: April 8, 2021 1 Orthogonality and Orthonormality
    Mathematical Toolkit Spring 2021 Lecture 4: April 8, 2021 Lecturer: Avrim Blum (notes based on notes from Madhur Tulsiani) 1 Orthogonality and orthonormality Definition 1.1 Two vectors u, v in an inner product space are said to be orthogonal if hu, vi = 0. A set of vectors S ⊆ V is said to consist of mutually orthogonal vectors if hu, vi = 0 for all u 6= v, u, v 2 S. A set of S ⊆ V is said to be orthonormal if hu, vi = 0 for all u 6= v, u, v 2 S and kuk = 1 for all u 2 S. Proposition 1.2 A set S ⊆ V n f0V g consisting of mutually orthogonal vectors is linearly inde- pendent. Proposition 1.3 (Gram-Schmidt orthogonalization) Given a finite set fv1,..., vng of linearly independent vectors, there exists a set of orthonormal vectors fw1,..., wng such that Span (fw1,..., wng) = Span (fv1,..., vng) . Proof: By induction. The case with one vector is trivial. Given the statement for k vectors and orthonormal fw1,..., wkg such that Span (fw1,..., wkg) = Span (fv1,..., vkg) , define k u + u = v − hw , v i · w and w = k 1 . k+1 k+1 ∑ i k+1 i k+1 k k i=1 uk+1 We can now check that the set fw1,..., wk+1g satisfies the required conditions. Unit length is clear, so let’s check orthogonality: k uk+1, wj = vk+1, wj − ∑ hwi, vk+1i · wi, wj = vk+1, wj − wj, vk+1 = 0. i=1 Corollary 1.4 Every finite dimensional inner product space has an orthonormal basis.
    [Show full text]
  • Orthogonal Complements (Revised Version)
    Orthogonal Complements (Revised Version) Math 108A: May 19, 2010 John Douglas Moore 1 The dot product You will recall that the dot product was discussed in earlier calculus courses. If n x = (x1: : : : ; xn) and y = (y1: : : : ; yn) are elements of R , we define their dot product by x · y = x1y1 + ··· + xnyn: The dot product satisfies several key axioms: 1. it is symmetric: x · y = y · x; 2. it is bilinear: (ax + x0) · y = a(x · y) + x0 · y; 3. and it is positive-definite: x · x ≥ 0 and x · x = 0 if and only if x = 0. The dot product is an example of an inner product on the vector space V = Rn over R; inner products will be treated thoroughly in Chapter 6 of [1]. Recall that the length of an element x 2 Rn is defined by p jxj = x · x: Note that the length of an element x 2 Rn is always nonnegative. Cauchy-Schwarz Theorem. If x 6= 0 and y 6= 0, then x · y −1 ≤ ≤ 1: (1) jxjjyj Sketch of proof: If v is any element of Rn, then v · v ≥ 0. Hence (x(y · y) − y(x · y)) · (x(y · y) − y(x · y)) ≥ 0: Expanding using the axioms for dot product yields (x · x)(y · y)2 − 2(x · y)2(y · y) + (x · y)2(y · y) ≥ 0 or (x · x)(y · y)2 ≥ (x · y)2(y · y): 1 Dividing by y · y, we obtain (x · y)2 jxj2jyj2 ≥ (x · y)2 or ≤ 1; jxj2jyj2 and (1) follows by taking the square root.
    [Show full text]
  • Orthogonality Handout
    3.8 (SUPPLEMENT) | ORTHOGONALITY OF EIGENFUNCTIONS We now develop some properties of eigenfunctions, to be used in Chapter 9 for Fourier Series and Partial Differential Equations. 1. Definition of Orthogonality R b We say functions f(x) and g(x) are orthogonal on a < x < b if a f(x)g(x) dx = 0 . [Motivation: Let's approximate the integral with a Riemann sum, as follows. Take a large integer N, put h = (b − a)=N and partition the interval a < x < b by defining x1 = a + h; x2 = a + 2h; : : : ; xN = a + Nh = b. Then Z b f(x)g(x) dx ≈ f(x1)g(x1)h + ··· + f(xN )g(xN )h a = (uN · vN )h where uN = (f(x1); : : : ; f(xN )) and vN = (g(x1); : : : ; g(xN )) are vectors containing the values of f and g. The vectors uN and vN are said to be orthogonal (or perpendicular) if their dot product equals zero (uN ·vN = 0), and so when we let N ! 1 in the above formula it makes R b sense to say the functions f and g are orthogonal when the integral a f(x)g(x) dx equals zero.] R π 1 2 π Example. sin x and cos x are orthogonal on −π < x < π, since −π sin x cos x dx = 2 sin x −π = 0. 2. Integration Lemma Suppose functions Xn(x) and Xm(x) satisfy the differential equations 00 Xn + λnXn = 0; a < x < b; 00 Xm + λmXm = 0; a < x < b; for some numbers λn; λm. Then Z b 0 0 b (λn − λm) Xn(x)Xm(x) dx = [Xn(x)Xm(x) − Xn(x)Xm(x)]a: a Proof.
    [Show full text]
  • Inner Products and Orthogonality
    Advanced Linear Algebra – Week 5 Inner Products and Orthogonality This week we will learn about: • Inner products (and the dot product again), • The norm induced by the inner product, • The Cauchy–Schwarz and triangle inequalities, and • Orthogonality. Extra reading and watching: • Sections 1.3.4 and 1.4.1 in the textbook • Lecture videos 17, 18, 19, 20, 21, and 22 on YouTube • Inner product space at Wikipedia • Cauchy–Schwarz inequality at Wikipedia • Gram–Schmidt process at Wikipedia Extra textbook problems: ? 1.3.3, 1.3.4, 1.4.1 ?? 1.3.9, 1.3.10, 1.3.12, 1.3.13, 1.4.2, 1.4.5(a,d) ??? 1.3.11, 1.3.14, 1.3.15, 1.3.25, 1.4.16 A 1.3.18 1 Advanced Linear Algebra – Week 5 2 There are many times when we would like to be able to talk about the angle between vectors in a vector space V, and in particular orthogonality of vectors, just like we did in Rn in the previous course. This requires us to have a generalization of the dot product to arbitrary vector spaces. Definition 5.1 — Inner Product Suppose that F = R or F = C, and V is a vector space over F. Then an inner product on V is a function h·, ·i : V × V → F such that the following three properties hold for all c ∈ F and all v, w, x ∈ V: a) hv, wi = hw, vi (conjugate symmetry) b) hv, w + cxi = hv, wi + chv, xi (linearity in 2nd entry) c) hv, vi ≥ 0, with equality if and only if v = 0.
    [Show full text]
  • LINEAR ALGEBRA FALL 2007/08 PROBLEM SET 9 SOLUTIONS In
    MATH 110: LINEAR ALGEBRA FALL 2007/08 PROBLEM SET 9 SOLUTIONS In the following V will denote a finite-dimensional vector space over R that is also an inner product space with inner product denoted by h ·; · i. The norm induced by h ·; · i will be denoted k · k. 1. Let S be a subset (not necessarily a subspace) of V . We define the orthogonal annihilator of S, denoted S?, to be the set S? = fv 2 V j hv; wi = 0 for all w 2 Sg: (a) Show that S? is always a subspace of V . ? Solution. Let v1; v2 2 S . Then hv1; wi = 0 and hv2; wi = 0 for all w 2 S. So for any α; β 2 R, hαv1 + βv2; wi = αhv1; wi + βhv2; wi = 0 ? for all w 2 S. Hence αv1 + βv2 2 S . (b) Show that S ⊆ (S?)?. Solution. Let w 2 S. For any v 2 S?, we have hv; wi = 0 by definition of S?. Since this is true for all v 2 S? and hw; vi = hv; wi, we see that hw; vi = 0 for all v 2 S?, ie. w 2 S?. (c) Show that span(S) ⊆ (S?)?. Solution. Since (S?)? is a subspace by (a) (it is the orthogonal annihilator of S?) and S ⊆ (S?)? by (b), we have span(S) ⊆ (S?)?. ? ? (d) Show that if S1 and S2 are subsets of V and S1 ⊆ S2, then S2 ⊆ S1 . ? Solution. Let v 2 S2 . Then hv; wi = 0 for all w 2 S2 and so for all w 2 S1 (since ? S1 ⊆ S2).
    [Show full text]
  • Eigenvalues, Eigenvectors and the Similarity Transformation
    Eigenvalues, Eigenvectors and the Similarity Transformation Eigenvalues and the associated eigenvectors are ‘special’ properties of square matrices. While the eigenvalues parameterize the dynamical properties of the system (timescales, resonance properties, amplification factors, etc) the eigenvectors define the vector coordinates of the normal modes of the system. Each eigenvector is associated with a particular eigenvalue. The general state of the system can be expressed as a linear combination of eigenvectors. The beauty of eigenvectors is that (for square symmetric matrices) they can be made orthogonal (decoupled from one another). The normal modes can be handled independently and an orthogonal expansion of the system is possible. The decoupling is also apparent in the ability of the eigenvectors to diagonalize the original matrix, A, with the eigenvalues lying on the diagonal of the new matrix, . In analogy to the inertia tensor in mechanics, the eigenvectors form the principle axes of the solid object and a similarity transformation rotates the coordinate system into alignment with the principle axes. Motion along the principle axes is decoupled. The matrix mechanics is closely related to the more general singular value decomposition. We will use the basis sets of orthogonal eigenvectors generated by SVD for orbit control problems. Here we develop eigenvector theory since it is more familiar to most readers. Square matrices have an eigenvalue/eigenvector equation with solutions that are the eigenvectors xand the associated eigenvalues : Ax = x The special property of an eigenvector is that it transforms into a scaled version of itself under the operation of A. Note that the eigenvector equation is non-linear in both the eigenvalue () and the eigenvector (x).
    [Show full text]
  • A Guided Tour to the Plane-Based Geometric Algebra PGA
    A Guided Tour to the Plane-Based Geometric Algebra PGA Leo Dorst University of Amsterdam Version 1.15{ July 6, 2020 Planes are the primitive elements for the constructions of objects and oper- ators in Euclidean geometry. Triangulated meshes are built from them, and reflections in multiple planes are a mathematically pure way to construct Euclidean motions. A geometric algebra based on planes is therefore a natural choice to unify objects and operators for Euclidean geometry. The usual claims of `com- pleteness' of the GA approach leads us to hope that it might contain, in a single framework, all representations ever designed for Euclidean geometry - including normal vectors, directions as points at infinity, Pl¨ucker coordinates for lines, quaternions as 3D rotations around the origin, and dual quaternions for rigid body motions; and even spinors. This text provides a guided tour to this algebra of planes PGA. It indeed shows how all such computationally efficient methods are incorporated and related. We will see how the PGA elements naturally group into blocks of four coordinates in an implementation, and how this more complete under- standing of the embedding suggests some handy choices to avoid extraneous computations. In the unified PGA framework, one never switches between efficient representations for subtasks, and this obviously saves any time spent on data conversions. Relative to other treatments of PGA, this text is rather light on the mathematics. Where you see careful derivations, they involve the aspects of orientation and magnitude. These features have been neglected by authors focussing on the mathematical beauty of the projective nature of the algebra.
    [Show full text]
  • Inner Product Spaces
    CHAPTER 6 Woman teaching geometry, from a fourteenth-century edition of Euclid’s geometry book. Inner Product Spaces In making the definition of a vector space, we generalized the linear structure (addition and scalar multiplication) of R2 and R3. We ignored other important features, such as the notions of length and angle. These ideas are embedded in the concept we now investigate, inner products. Our standing assumptions are as follows: 6.1 Notation F, V F denotes R or C. V denotes a vector space over F. LEARNING OBJECTIVES FOR THIS CHAPTER Cauchy–Schwarz Inequality Gram–Schmidt Procedure linear functionals on inner product spaces calculating minimum distance to a subspace Linear Algebra Done Right, third edition, by Sheldon Axler 164 CHAPTER 6 Inner Product Spaces 6.A Inner Products and Norms Inner Products To motivate the concept of inner prod- 2 3 x1 , x 2 uct, think of vectors in R and R as x arrows with initial point at the origin. x R2 R3 H L The length of a vector in or is called the norm of x, denoted x . 2 k k Thus for x .x1; x2/ R , we have The length of this vector x is p D2 2 2 x x1 x2 . p 2 2 x1 x2 . k k D C 3 C Similarly, if x .x1; x2; x3/ R , p 2D 2 2 2 then x x1 x2 x3 . k k D C C Even though we cannot draw pictures in higher dimensions, the gener- n n alization to R is obvious: we define the norm of x .x1; : : : ; xn/ R D 2 by p 2 2 x x1 xn : k k D C C The norm is not linear on Rn.
    [Show full text]
  • Math 480 Notes on Orthogonality the Word Orthogonal Is a Synonym for Perpendicular. Question 1: When Are Two Vectors V 1 and V2
    Math 480 Notes on Orthogonality The word orthogonal is a synonym for perpendicular. n Question 1: When are two vectors ~v1 and ~v2 in R orthogonal to one another? The most basic answer is \if the angle between them is 90◦" but this is not very practical. How could you tell whether the vectors 0 1 1 0 1 1 @ 1 A and @ 3 A 1 1 are at 90◦ from one another? One way to think about this is as follows: ~v1 and ~v2 are orthogonal if and only if the triangle formed by ~v1, ~v2, and ~v1 − ~v2 (drawn with its tail at ~v2 and its head at ~v1) is a right triangle. The Pythagorean Theorem then tells us that this triangle is a right triangle if and only if 2 2 2 (1) jj~v1jj + jj~v2jj = jj~v1 − ~v2jj ; where jj − jj denotes the length of a vector. 0 x1 1 . The length of a vector ~x = @ . A is easy to measure: the Pythagorean Theorem (once again) xn tells us that q 2 2 jj~xjj = x1 + ··· + xn: This expression under the square root is simply the matrix product 0 x1 1 T . ~x ~x = (x1 ··· xn) @ . A : xn Definition. The inner product (also called the dot product) of two vectors ~x;~y 2 Rn, written h~x;~yi or ~x · ~y, is defined by n T X hx; yi = ~x ~y = xiyi: i=1 Since matrix multiplication is linear, inner products satisfy h~x;~y1 + ~y2i = h~x;~y1i + h~x;~y2i h~x1; a~yi = ah~x1; ~yi: (Similar formulas hold in the first coordinate, since h~x;~yi = h~y; ~xi.) Now we can write 2 2 jj~v1 − ~v2jj = h~v1 − ~v2;~v1 − ~v2i = h~v1;~v1i − 2h~v1;~v2i + h~v2;~v2i = jj~v1jj − 2h~v1;~v2i + jj~v2jj; so Equation (1) holds if and only if h~v1;~v2i = 0: n Answer to Question 1: Vectors ~v1 and ~v2 in R are orthogonal if and only if h~v1;~v2i = 0.
    [Show full text]