DOCSLIB.ORG

Linear Algebra Review Concepts1 Orthogonality • Least-Squares: Let a Be an M

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Linear Algebra Review Concepts1 Orthogonality • Least-Squares: Let a Be an M Linear Algebra review concepts1 Orthogonality • Let W be a subspace of Rn. For any v 2 Rn, the projection of v onto Rn, denoted v^, is the point in W closest to v. Let n and be an orthogonal basis of . Then the projection of n onto is • R fw1; w2; :::; wkg W v 2 R W w1 · v w2 · v wk · v v^ = w1 + w2 + ::: + wk: w1 · w1 w2 · w2 wk · wk • v^ is the point in W closest to v. • If v 2 W then v^ = v. If v ? W then v^ = 0. For any v, we can write v = v^ + v? where v? = v − v^ 2 W ?. The projection matrix for projection onto is 1 T . • spanfwg P = w·w ww Gram-Schmidt: Given a subspace n and a basis of , we can nd an orthogonal basis • W ⊂ R fa1; :::; akg W fb1; :::; bkg of W and an orthonormal basis fu1; :::; ukg of W . The algorithm is as follows: b1 1. b1 = a1. Then u1 = . kb1k b2 2. b2 = a2 − (u1 · a2)u1. Then u2 = . kb2k . bj i. bj = aj − (u1 · aj)u1 − ::: − (uj−1 · aj)uj−1. Then uj = . kbj k • Least-Squares: Let A be an m × n matrix and b 2 Rm. The least squares solution of Ax = b, denoted x^, is the vector that minimizes the error Ax^ − b. Note that if Ax = b has a solution, then x^ = x. • Least-Squares can be solved in two ways: 1. Ax^ = b^. Compute b^, the orthogonal projection of b onto col(A), then solve for x^. 2. AT Ax^ = AT b. Compute AT A and AT b, then solve for x^. In general, this method is easier! Furthermore, after you have found x^, it is easy to compute b^ = Ax^. • For a matrix A with linearly independent columns, the matrix that represents projection onto col(A) is P = A(AT A)−1AT : • If Q is an m × n matrix with orthonormal columns, then QT Q = I. If in addition Q is n × n (we call Q an orthogonal matrix), then Q−1 = QT . • If Q has orthonormal columns, then the matrix that represents projection onto col(Q) is P = QQT . Note: if Q is n × n, then because Q−1 = QT ;P = QQT = I. I.e., the projection matrix onto col(Q) is the identity matrix. Why does this make sense? What is col(Q)? • QR Decomposition Let A be an m×n matrix with linearly independent columns. Then we can write A = QR where Q is an m × n matrix with orthonormal columns and R is n × n, upper triangular and invertible. 2 . 3 . 6 7 • To compute Q, let A = 6 a1 a2 ::: an 7 and apply Gram-Schmidt process to fa1; :::; ang to get an 4 . 5 . 2 . 3 . 6 7 orthonormal basis fq1; :::; qng of col(A). Then Q = 6 q1 q2 ::: qn 7 and Q is an orthogonal matrix. 4 . 5 . Note that QT A = QT QR = Q−1QR = R: Therefore to nd R, we simply compute R = QT A. 1Email: [email protected]; there may be typos. 1 Determinants • Let A be an n × n matrix. The determinant of A, det(A), is a scalar that satises the following: If A is upper or lower triangular, then det(A) is the product of diagonal entries. We can reduce any matrix to upper triangular form (echelon form) using row operations. The row operations aect the determinant in the following way: A ! B by sending Ri ! Ri + αRj. Then det(A) = det(B). A ! B by swapping Ri $ Rj. Then −det(A) = det(B). A ! B by scaling Ri ! αRi (α 6= 0). Then α · det(A) = det(B). We can use the above method to nd the determinant of a square matrix of any size. • A is invertible if and only if det(A) 6= 0. a b • For 2 × 2 matrices ONLY: if A = , then det(A) = ad − bc. c d 2 a b c 3 • For 3 × 3 matrices ONLY: if A = 4 d e f 5 then det(A) = [aei + bfg + cdh] − [ceg + bdi + afh]. g h i (Add products of left to right diagonals, subtract products of right to left diagonals. This is similar to computing cross products.) • Cofactor Expansion: 2 3 a1;1 ··· a1;n 6 a2;1 ··· a2;n 7 (For square matrices of any size!) Let A = 6 7. Pick any row or column of A to expand 4 ··· 5 an;1 ··· an;n along. Dene Ai;j to be the determinant of the (n − 1) × (n − 1) matrix obtained by removing the ith row and jth column of A. If we choose to expand along the ith row, then i+1 i+2 i+n det(A) = (−1) ai;1Ai;1 + (−1) ai;2Ai;2 + ::: + (−1) ai;nAi;n; if instead the jth column is chosen, then 1+j 2+j n+j det(A) = (−1) a1;jA1;j + (−1) a2;jA2;j + ::: + (−1) an;jAn;j: 2 + − + ··· 3 6 − + − · · · 7 You may nd the following checkerboard matrix helpful for determining the sign of each term: 6 7. n×n 6 + − + ··· 7 4 . 5 . 2 1 2 3 3 For example, let A = 4 4 5 6 5. If I choose to expand along the second row then: 7 8 9 2 3 1 3 1 2 det(A) = −4det + 5det − 6det : 8 9 7 9 7 8 Note: the cofactor expansion method is generally more computation than other methods. However, if we have a row or column with many zero entries, then we can expand along that row or column and reduce the amount of computation required. • If A; B are n × n matrices, then det(AB) = det(A) · det(B). In particular, if is invertible (therefore ) then −1 1 . • A det(A) 6= 0 det(A ) = det(A) • det(AT ) = det(A). • If A is n × n, then for any scalar α, det(αA) = αndet(A). 2 Eigenvalues and Eigenvectors • Let A be an n × n matrix. x where x 6= 0 is an eigenvector of A if Ax = λx for some scalar λ. We call λ an eigenvalue of A, and x is an eigenvector corresponding to λ. The eigenspace corresponding to λ is the subspace of the eigenvectors corresponding to λ. • The characteristic polynomial of A is p(λ) = det(A − λI). • To nd the eigenvalues of A, nd the roots of p(λ), i.e., solve for λ such that det(A − λI) = 0. • Given λ, to nd the eigenvectors (and eigenspace) corresponding to λ, nd nul(A − λI). • Any nonzero vector in the null space of A is an eigenvector of A corresponding to λ = 0. Thus A is noninvertible if and only if 0 is an eigenvalue of A. • If fx1; :::; xkg are eigenvectors of A corresponding to the same eigenvalue λ, then any linear combination of fx1; :::; xkg is also an eigenvector of A corresponding to λ. • If A is an n × n matrix, then A has n eigenvalues (counted with multiplicity). The eigenvalues may be real or complex. • The determinant of A is equal to the product of eigenvalues of A. • The trace of A (sum of diagonal entries) is equal to the sum of the eigenvalues of A. • A has at most n linearly independent eigenvectors. • If λ is an eigenvalue of A with multiplicity k (i.e., λ appears as a root of p(λ) k times), then A has at most k linearly independent eigenvectors corresponding to λ. Discrete Dynamical Systems Let n be a vector that changes with time where for some matrix (we call the • xt 2 R t xt+1 = Axt n × n A A 2 transition matrix). Then x1 = Ax0; x2 = Ax1 = A(Ax0) = A x0 and in general, k xk = A x0 where x0 is the initial condition. • A Markov process is a discrete dynamical system with properties: the total population remains xed and the population of a given state can never be negative. • Within the population, we have n dierent classes, call them class 1; 2; :::; n. At every time step, there is a probability that entities from one class can move to another class (or stay in the same class). We wish to nd the equilibrium state of the classes as t ! 1. • As in discrete dynamical systems, we can model a Markov process by xt+1 = Axt where A is an n × n matrix 2 3 x1;t . and 6 . 7 reports the population at time . is the percentage of class in the total population at xt = 4 . 5 t xi;t i xn;t time t. • We call A a Markov matrix. A has the following properties: the columns of A add up to 1, no entry of A is negative, 1 is always an eigenvalue of A and all other eigenvalues are less than 1 in absolute value. • We wish to nd the equilibrium state as t ! 1, call it x1. Observe Ax1 = lim Axn = lim xn+1 = x1; n!1 n!1 therefore to nd the equilibrium state x1, we should look at the eigenvector corresponding toλ = 1. 3 • The sum of the entries of the equilibrium vector should be 1 since we are talking about the sum of percentages that add up to a whole. Note: this is not a unit vector! • Let fv1; v2; :::; vng be an eigenbasis of the Markov matrix A (with corresponding eigenvalues λ1(= 1); λ2; :::; λn respectively). For any vector x, we can write x = c1v1 + ::: + cnvn for some scalars c1; :::; cn.
Load more

Recommended publications
  • A New Description of Space and Time Using Clifford Multivectors
    A new description of space and time using Clifford multivectors James M. Chappell† , Nicolangelo Iannella† , Azhar Iqbal† , Mark Chappell‡ , Derek Abbott† †School of Electrical and Electronic Engineering, University of Adelaide, South Australia 5005, Australia ‡Griffith Institute, Griffith University, Queensland 4122, Australia Abstract Following the development of the special theory of relativity in 1905, Minkowski pro- posed a unified space and time structure consisting of three space dimensions and one time dimension, with relativistic effects then being natural consequences of this space- time geometry. In this paper, we illustrate how Clifford’s geometric algebra that utilizes multivectors to represent spacetime, provides an elegant mathematical framework for the study of relativistic phenomena. We show, with several examples, how the application of geometric algebra leads to the correct relativistic description of the physical phenomena being considered. This approach not only provides a compact mathematical representa- tion to tackle such phenomena, but also suggests some novel insights into the nature of time. Keywords: Geometric algebra, Clifford space, Spacetime, Multivectors, Algebraic framework 1. Introduction The physical world, based on early investigations, was deemed to possess three inde- pendent freedoms of translation, referred to as the three dimensions of space. This naive conclusion is also supported by more sophisticated analysis such as the existence of only five regular polyhedra and the inverse square force laws. If we lived in a world with four spatial dimensions, for example, we would be able to construct six regular solids, and in arXiv:1205.5195v2 [math-ph] 11 Oct 2012 five dimensions and above we would find only three [1].
    [Show full text]
  • 21. Orthonormal Bases
    21. Orthonormal Bases The canonical/standard basis 011 001 001 B C B C B C B0C B1C B0C e1 = B.C ; e2 = B.C ; : : : ; en = B.C B.C B.C B.C @.A @.A @.A 0 0 1 has many useful properties. • Each of the standard basis vectors has unit length: q p T jjeijj = ei ei = ei ei = 1: • The standard basis vectors are orthogonal (in other words, at right angles or perpendicular). T ei ej = ei ej = 0 when i 6= j This is summarized by ( 1 i = j eT e = δ = ; i j ij 0 i 6= j where δij is the Kronecker delta. Notice that the Kronecker delta gives the entries of the identity matrix. Given column vectors v and w, we have seen that the dot product v w is the same as the matrix multiplication vT w. This is the inner product on n T R . We can also form the outer product vw , which gives a square matrix. 1 The outer product on the standard basis vectors is interesting. Set T Π1 = e1e1 011 B C B0C = B.C 1 0 ::: 0 B.C @.A 0 01 0 ::: 01 B C B0 0 ::: 0C = B. .C B. .C @. .A 0 0 ::: 0 . T Πn = enen 001 B C B0C = B.C 0 0 ::: 1 B.C @.A 1 00 0 ::: 01 B C B0 0 ::: 0C = B. .C B. .C @. .A 0 0 ::: 1 In short, Πi is the diagonal square matrix with a 1 in the ith diagonal position and zeros everywhere else.
    [Show full text]
  • Partitioned (Or Block) Matrices This Version: 29 Nov 2018
    Partitioned (or Block) Matrices This version: 29 Nov 2018 Intermediate Econometrics / Forecasting Class Notes Instructor: Anthony Tay It is frequently convenient to partition matrices into smaller sub-matrices. e.g. 2 3 2 1 3 2 3 2 1 3 4 1 1 0 7 4 1 1 0 7 A B (2×2) (2×3) 3 1 1 0 0 = 3 1 1 0 0 = C I 1 3 0 1 0 1 3 0 1 0 (3×2) (3×3) 2 0 0 0 1 2 0 0 0 1 The same matrix can be partitioned in several different ways. For instance, we can write the previous matrix as 2 3 2 1 3 2 3 2 1 3 4 1 1 0 7 4 1 1 0 7 a b0 (1×1) (1×4) 3 1 1 0 0 = 3 1 1 0 0 = c D 1 3 0 1 0 1 3 0 1 0 (4×1) (4×4) 2 0 0 0 1 2 0 0 0 1 One reason partitioning is useful is that we can do matrix addition and multiplication with blocks, as though the blocks are elements, as long as the blocks are conformable for the operations. For instance: A B D E A + D B + E (2×2) (2×3) (2×2) (2×3) (2×2) (2×3) + = C I C F 2C I + F (3×2) (3×3) (3×2) (3×3) (3×2) (3×3) A B d E Ad + BF AE + BG (2×2) (2×3) (2×1) (2×3) (2×1) (2×3) = C I F G Cd + F CE + G (3×2) (3×3) (3×1) (3×3) (3×1) (3×3) | {z } | {z } | {z } (5×5) (5×4) (5×4) 1 Intermediate Econometrics / Forecasting 2 Examples (1) Let 1 2 1 1 2 1 c 1 4 2 3 4 2 3 h i A = = = a a a and c = c 1 2 3 2 3 0 1 3 0 1 c 0 1 3 0 1 3 3 c1 h i then Ac = a1 a2 a3 c2 = c1a1 + c2a2 + c3a3 c3 The product Ac produces a linear combination of the columns of A.
    [Show full text]
  • Graph Equivalence Classes for Spectral Projector-Based Graph Fourier Transforms Joya A
    1 Graph Equivalence Classes for Spectral Projector-Based Graph Fourier Transforms Joya A. Deri, Member, IEEE, and José M. F. Moura, Fellow, IEEE Abstract—We define and discuss the utility of two equiv- Consider a graph G = G(A) with adjacency matrix alence graph classes over which a spectral projector-based A 2 CN×N with k ≤ N distinct eigenvalues and Jordan graph Fourier transform is equivalent: isomorphic equiv- decomposition A = VJV −1. The associated Jordan alence classes and Jordan equivalence classes. Isomorphic equivalence classes show that the transform is equivalent subspaces of A are Jij, i = 1; : : : k, j = 1; : : : ; gi, up to a permutation on the node labels. Jordan equivalence where gi is the geometric multiplicity of eigenvalue 휆i, classes permit identical transforms over graphs of noniden- or the dimension of the kernel of A − 휆iI. The signal tical topologies and allow a basis-invariant characterization space S can be uniquely decomposed by the Jordan of total variation orderings of the spectral components. subspaces (see [13], [14] and Section II). For a graph Methods to exploit these classes to reduce computation time of the transform as well as limitations are discussed. signal s 2 S, the graph Fourier transform (GFT) of [12] is defined as Index Terms—Jordan decomposition, generalized k gi eigenspaces, directed graphs, graph equivalence classes, M M graph isomorphism, signal processing on graphs, networks F : S! Jij i=1 j=1 s ! (s ;:::; s ;:::; s ;:::; s ) ; (1) b11 b1g1 bk1 bkgk I. INTRODUCTION where sij is the (oblique) projection of s onto the Jordan subspace Jij parallel to SnJij.
    [Show full text]
  • Handout 9 More Matrix Properties; the Transpose
    Handout 9 More matrix properties; the transpose Square matrix properties These properties only apply to a square matrix, i.e. n £ n. ² The leading diagonal is the diagonal line consisting of the entries a11, a22, a33, . ann. ² A diagonal matrix has zeros everywhere except the leading diagonal. ² The identity matrix I has zeros o® the leading diagonal, and 1 for each entry on the diagonal. It is a special case of a diagonal matrix, and A I = I A = A for any n £ n matrix A. ² An upper triangular matrix has all its non-zero entries on or above the leading diagonal. ² A lower triangular matrix has all its non-zero entries on or below the leading diagonal. ² A symmetric matrix has the same entries below and above the diagonal: aij = aji for any values of i and j between 1 and n. ² An antisymmetric or skew-symmetric matrix has the opposite entries below and above the diagonal: aij = ¡aji for any values of i and j between 1 and n. This automatically means the digaonal entries must all be zero. Transpose To transpose a matrix, we reect it across the line given by the leading diagonal a11, a22 etc. In general the result is a di®erent shape to the original matrix: a11 a21 a11 a12 a13 > > A = A = 0 a12 a22 1 [A ]ij = A : µ a21 a22 a23 ¶ ji a13 a23 @ A > ² If A is m £ n then A is n £ m. > ² The transpose of a symmetric matrix is itself: A = A (recalling that only square matrices can be symmetric).
    [Show full text]
  • Lecture 4: April 8, 2021 1 Orthogonality and Orthonormality
    Mathematical Toolkit Spring 2021 Lecture 4: April 8, 2021 Lecturer: Avrim Blum (notes based on notes from Madhur Tulsiani) 1 Orthogonality and orthonormality Definition 1.1 Two vectors u, v in an inner product space are said to be orthogonal if hu, vi = 0. A set of vectors S ⊆ V is said to consist of mutually orthogonal vectors if hu, vi = 0 for all u 6= v, u, v 2 S. A set of S ⊆ V is said to be orthonormal if hu, vi = 0 for all u 6= v, u, v 2 S and kuk = 1 for all u 2 S. Proposition 1.2 A set S ⊆ V n f0V g consisting of mutually orthogonal vectors is linearly inde- pendent. Proposition 1.3 (Gram-Schmidt orthogonalization) Given a finite set fv1,..., vng of linearly independent vectors, there exists a set of orthonormal vectors fw1,..., wng such that Span (fw1,..., wng) = Span (fv1,..., vng) . Proof: By induction. The case with one vector is trivial. Given the statement for k vectors and orthonormal fw1,..., wkg such that Span (fw1,..., wkg) = Span (fv1,..., vkg) , define k u + u = v − hw , v i · w and w = k 1 . k+1 k+1 ∑ i k+1 i k+1 k k i=1 uk+1 We can now check that the set fw1,..., wk+1g satisfies the required conditions. Unit length is clear, so let’s check orthogonality: k uk+1, wj = vk+1, wj − ∑ hwi, vk+1i · wi, wj = vk+1, wj − wj, vk+1 = 0. i=1 Corollary 1.4 Every finite dimensional inner product space has an orthonormal basis.
    [Show full text]
  • EUCLIDEAN DISTANCE MATRIX COMPLETION PROBLEMS June 6
    EUCLIDEAN DISTANCE MATRIX COMPLETION PROBLEMS HAW-REN FANG∗ AND DIANNE P. O’LEARY† June 6, 2010 Abstract. A Euclidean distance matrix is one in which the (i, j) entry specifies the squared distance between particle i and particle j. Given a partially-specified symmetric matrix A with zero diagonal, the Euclidean distance matrix completion problem (EDMCP) is to determine the unspecified entries to make A a Euclidean distance matrix. We survey three different approaches to solving the EDMCP. We advocate expressing the EDMCP as a nonconvex optimization problem using the particle positions as variables and solving using a modified Newton or quasi-Newton method. To avoid local minima, we develop a randomized initial- ization technique that involves a nonlinear version of the classical multidimensional scaling, and a dimensionality relaxation scheme with optional weighting. Our experiments show that the method easily solves the artificial problems introduced by Mor´e and Wu. It also solves the 12 much more difficult protein fragment problems introduced by Hen- drickson, and the 6 larger protein problems introduced by Grooms, Lewis, and Trosset. Key words. distance geometry, Euclidean distance matrices, global optimization, dimensional- ity relaxation, modified Cholesky factorizations, molecular conformation AMS subject classifications. 49M15, 65K05, 90C26, 92E10 1. Introduction. Given the distances between each pair of n particles in Rr, n r, it is easy to determine the relative positions of the particles. In many applications,≥ though, we are given only some of the distances and we would like to determine the missing distances and thus the particle positions. We focus in this paper on algorithms to solve this distance completion problem.
    [Show full text]
  • Week 8-9. Inner Product Spaces. (Revised Version) Section 3.1 Dot Product As an Inner Product
    Math 2051 W2008 Margo Kondratieva Week 8-9. Inner product spaces. (revised version) Section 3.1 Dot product as an inner product. Consider a linear (vector) space V . (Let us restrict ourselves to only real spaces that is we will not deal with complex numbers and vectors.) De¯nition 1. An inner product on V is a function which assigns a real number, denoted by < ~u;~v> to every pair of vectors ~u;~v 2 V such that (1) < ~u;~v>=< ~v; ~u> for all ~u;~v 2 V ; (2) < ~u + ~v; ~w>=< ~u;~w> + < ~v; ~w> for all ~u;~v; ~w 2 V ; (3) < k~u;~v>= k < ~u;~v> for any k 2 R and ~u;~v 2 V . (4) < ~v;~v>¸ 0 for all ~v 2 V , and < ~v;~v>= 0 only for ~v = ~0. De¯nition 2. Inner product space is a vector space equipped with an inner product. Pn It is straightforward to check that the dot product introduces by ~u ¢ ~v = j=1 ujvj is an inner product. You are advised to verify all the properties listed in the de¯nition, as an exercise. The dot product is also called Euclidian inner product. De¯nition 3. Euclidian vector space is Rn equipped with Euclidian inner product < ~u;~v>= ~u¢~v. De¯nition 4. A square matrix A is called positive de¯nite if ~vT A~v> 0 for any vector ~v 6= ~0. · ¸ 2 0 Problem 1. Show that is positive de¯nite. 0 3 Solution: Take ~v = (x; y)T . Then ~vT A~v = 2x2 + 3y2 > 0 for (x; y) 6= (0; 0).
    [Show full text]
  • New Foundations for Geometric Algebra1
    Text published in the electronic journal Clifford Analysis, Clifford Algebras and their Applications vol. 2, No. 3 (2013) pp. 193-211 New foundations for geometric algebra1 Ramon González Calvet Institut Pere Calders, Campus Universitat Autònoma de Barcelona, 08193 Cerdanyola del Vallès, Spain E-mail : [email protected] Abstract. New foundations for geometric algebra are proposed based upon the existing isomorphisms between geometric and matrix algebras. Each geometric algebra always has a faithful real matrix representation with a periodicity of 8. On the other hand, each matrix algebra is always embedded in a geometric algebra of a convenient dimension. The geometric product is also isomorphic to the matrix product, and many vector transformations such as rotations, axial symmetries and Lorentz transformations can be written in a form isomorphic to a similarity transformation of matrices. We collect the idea Dirac applied to develop the relativistic electron equation when he took a basis of matrices for the geometric algebra instead of a basis of geometric vectors. Of course, this way of understanding the geometric algebra requires new definitions: the geometric vector space is defined as the algebraic subspace that generates the rest of the matrix algebra by addition and multiplication; isometries are simply defined as the similarity transformations of matrices as shown above, and finally the norm of any element of the geometric algebra is defined as the nth root of the determinant of its representative matrix of order n. The main idea of this proposal is an arithmetic point of view consisting of reversing the roles of matrix and geometric algebras in the sense that geometric algebra is a way of accessing, working and understanding the most fundamental conception of matrix algebra as the algebra of transformations of multiple quantities.
    [Show full text]
  • Matrix Determinants
    MATRIX DETERMINANTS Summary Uses ................................................................................................................................................. 1 1‐ Reminder ‐ Definition and components of a matrix ................................................................ 1 2‐ The matrix determinant .......................................................................................................... 2 3‐ Calculation of the determinant for a matrix ................................................................. 2 4‐ Exercise .................................................................................................................................... 3 5‐ Definition of a minor ............................................................................................................... 3 6‐ Definition of a cofactor ............................................................................................................ 4 7‐ Cofactor expansion – a method to calculate the determinant ............................................... 4 8‐ Calculate the determinant for a matrix ........................................................................ 5 9‐ Alternative method to calculate determinants ....................................................................... 6 10‐ Exercise .................................................................................................................................... 7 11‐ Determinants of square matrices of dimensions 4x4 and greater ........................................
    [Show full text]
  • Orthogonality Handout
    3.8 (SUPPLEMENT) | ORTHOGONALITY OF EIGENFUNCTIONS We now develop some properties of eigenfunctions, to be used in Chapter 9 for Fourier Series and Partial Differential Equations. 1. Definition of Orthogonality R b We say functions f(x) and g(x) are orthogonal on a < x < b if a f(x)g(x) dx = 0 . [Motivation: Let's approximate the integral with a Riemann sum, as follows. Take a large integer N, put h = (b − a)=N and partition the interval a < x < b by defining x1 = a + h; x2 = a + 2h; : : : ; xN = a + Nh = b. Then Z b f(x)g(x) dx ≈ f(x1)g(x1)h + ··· + f(xN )g(xN )h a = (uN · vN )h where uN = (f(x1); : : : ; f(xN )) and vN = (g(x1); : : : ; g(xN )) are vectors containing the values of f and g. The vectors uN and vN are said to be orthogonal (or perpendicular) if their dot product equals zero (uN ·vN = 0), and so when we let N ! 1 in the above formula it makes R b sense to say the functions f and g are orthogonal when the integral a f(x)g(x) dx equals zero.] R π 1 2 π Example. sin x and cos x are orthogonal on −π < x < π, since −π sin x cos x dx = 2 sin x −π = 0. 2. Integration Lemma Suppose functions Xn(x) and Xm(x) satisfy the differential equations 00 Xn + λnXn = 0; a < x < b; 00 Xm + λmXm = 0; a < x < b; for some numbers λn; λm. Then Z b 0 0 b (λn − λm) Xn(x)Xm(x) dx = [Xn(x)Xm(x) − Xn(x)Xm(x)]a: a Proof.
    [Show full text]
  • Inner Products and Orthogonality
    Advanced Linear Algebra – Week 5 Inner Products and Orthogonality This week we will learn about: • Inner products (and the dot product again), • The norm induced by the inner product, • The Cauchy–Schwarz and triangle inequalities, and • Orthogonality. Extra reading and watching: • Sections 1.3.4 and 1.4.1 in the textbook • Lecture videos 17, 18, 19, 20, 21, and 22 on YouTube • Inner product space at Wikipedia • Cauchy–Schwarz inequality at Wikipedia • Gram–Schmidt process at Wikipedia Extra textbook problems: ? 1.3.3, 1.3.4, 1.4.1 ?? 1.3.9, 1.3.10, 1.3.12, 1.3.13, 1.4.2, 1.4.5(a,d) ??? 1.3.11, 1.3.14, 1.3.15, 1.3.25, 1.4.16 A 1.3.18 1 Advanced Linear Algebra – Week 5 2 There are many times when we would like to be able to talk about the angle between vectors in a vector space V, and in particular orthogonality of vectors, just like we did in Rn in the previous course. This requires us to have a generalization of the dot product to arbitrary vector spaces. Definition 5.1 — Inner Product Suppose that F = R or F = C, and V is a vector space over F. Then an inner product on V is a function h·, ·i : V × V → F such that the following three properties hold for all c ∈ F and all v, w, x ∈ V: a) hv, wi = hw, vi (conjugate symmetry) b) hv, w + cxi = hv, wi + chv, xi (linearity in 2nd entry) c) hv, vi ≥ 0, with equality if and only if v = 0.
    [Show full text]