DOCSLIB.ORG

Section 7.5 Inner Product Spaces with the “Dot Product” Defined In

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Section 7.5 Inner Product Spaces with the “Dot Product” Defined In Section 7.5 Inner Product Spaces With the “dot product” defined in Chapter 6, we were able to study the following properties of vectors in Rn. 1) Length or norm of a vector u. ( u = p u u ) || || · 2) Distance of two given vectors u and v. (||u - v||) 3) Orthogonality of two vectors u and v. (whether u v = 0) · Now, how do we describe the same properties of vectors in other types of vector spaces? For example, 1) How do we define the norm of the function f(x) in C([a, b])? 2) How do we determine whether the polynomials f(x) and g(x) in P2 are orthogonal? 3) How do we calculate the distance between the two matrices A and B in M4x3? 1 Section 7.5 Inner Product Spaces Definition. Property: Many inner products may be defined on a vector space. Proof If 〈·, ·〉 is an inner product, then 〈·, ·〉r defined by 〈u, v〉r = r〈u, v〉 is also an inner product for any r > 0. Definition. 2 Example: The dot product is an inner product on Rn. Definition. 3 Example: V = C([a, b]) = {f | f : [a, b] → R, f is continuous} is a vector space, and the function 〈·, ·〉 : V×V → R defined by b 〈 f , g〉 = f (t)g(t)dt ∫a ∀ f , g ∈ V is an inner product on V. Axiom 1: f 2 is continuous and non-negative. 2 f ≠ 0 ⇒ f (t0) > 0 for some t0 ∈ [a, b]. 2 ⇒ f (t) > p > 0 ∀ [t0-r/2, t0+r/2] ⊆ [a, b]. b ⇒ 〈 f , f 〉 = f 2 (t)dt ≥ r ⋅ p > 0. ∫a Axioms 2 - 4: You examine them. 4 Example: 〈A, B〉 = trace(ABT) is the Frobenius inner product on Rn×n. T 2 Axiom 1: 〈A, A〉 = trace(AA ) = Σ1≤i,j≤n(aij) > 0 ∀ A ≠ O. Axiom 2: trace(ABT) = trace(ABT)T = trace(BAT). Axioms 3 - 4: You examine them. 5 Definition. For any vector v in an inner product space V ,thenorm or length of v is denoted and defined as v = v, v 1/2.Thedistance between u, v V is defined as u v . || || h i 2 || − || n×n 2 1/2 Example: The Frobenius norm in R is ||A|| = [Σ1≤i,j≤n(aij) ] , since T 〈A, B〉 = trace(AB ) = Σ1≤i,j≤naijbij. Property: Inner products and norms satisfy the elementary properties stated in Theorem 6.1, the Cauchy-Schwarz inequality, and the triangle inequality in Section 6.1. Proof You show it. 6 Theorem 6.1 Let u and v be vectors in n and c be a scalar in . (a) u u = u 2. R R (b) u · u =|| 0 if|| and only if u = 0. (c) u ·v = v u. (d) u· (v + w·)=u v + u w. (e) (v·+ w) u = v ·u + w · u. (f) (cu) v =· c(u u·)=u (·cu). (g) cu · = c u ·. · || || | ||| || Let V be an inner product space and let u and v be vectors in V and c be a scalar in . It can be shown that (a) u, u R= u 2. (b) hu, ui =|| 0 if|| and only if u = 0. (c) hu, v i = v, u . (d) hu, vi+ wh = iu, v + u, w . (e) hv + w, ui = hv, ui+ hw, ui. (f) hcu, v = ic u,hu =i uh,cu .i (g) h cu i= c hu . i h i || || | ||| || 7 Theorem 6.2 (Pythagorean theorem in Rn) Let u and v be vectors in n.Thenu and v are orthogonal if and only if R u + v 2 = u 2 + v 2. || || || || || || Proof u + v 2 = u 2 +2u v + v 2 || || || || · || || = 0 if and only if u and v are orthogonal Pythagorean theorem in any inner product space V Let V be an inner product space and let u and v be vectors in V .Thenu and v are orthogonal if and only if u + v 2 = u 2 + v 2. || || || || || || u + v 2 = u 2 +2 u, v + v 2 Proof || || || || h i || || = 0 if and only if 8 u and v are orthogonal Theorem 6.3 (Cauchy-Schwarz inequality) For any vectors u and v in n,wehave R u v u v . | · ||| || · || || Proof Using Theorem 6.2. Cauchy-Schwarz inequality in any inner product space V Let V be an inner product space. For any vectors u and v in V ,wehave u, v u v . |h i||| || · || || Proof Using Pythagorean Theorem. Example: A Cauchy-Schwarz inequality in C([a, b]) 2 b b b f(t)g(t)dt f 2(t)dt g2(t)dt a a a 9 Z ! Z ! Z ! Theorem 6.4 (Triangle inequality) For any vectors u and v in n,wehave R u + v u + v . || || || || || || Proof Using Theorem 6.3. Triangle inequality in any inner product space V Let V be an inner product space. For any vectors u and v in V ,wehave u + v u + v . || || || || || || Proof Using Cauchy-Schwartz inequality. 10 Definition. In an inner product space V , the vectors u, v are called orthogonal if u, v = 0, a vector u is called a unit vector if u = 1, a subset S is called orthogonalh i if u, v = 0 for all distinct u, v S||, and|| S is called orthonormal if S is orthogonalh i and u = 1 for all u2 S. || || 2 Properties: 1. Every nonzero vector v in an inner product space may be changed into a unit normalized vector (1/||v||)v, and every orthogonal subset with only nonzero vectors may be changed into an orthonormal subset without affecting the subspace spanned. 2. An orthogonal set of nonzero vectors is L.I., no matter the set is finite or infinite. Example: In the inner product space C([0, 2π]), the vectors f (t) = sin 3t and g(t) = cos 2t are orthogonal, since 11 Example: In the vector space of trigonometric polynomials [0, 2⇡] = Span 1, cos t, sin t, cos 2t, sin 2t, , cos nt, sin nt, T { ··· ···} = Span S ⇒ S is orthogonal, since 2⇡ cos nt, sin mt = cos nt sin mtdt =0, n, m 0 h i 0 8 ≥ Z 2⇡ cos nt, cos mt = cos nt cos mtdt =0, n = m h i 0 8 6 Z 2⇡ sin nt, sin mt = sin nt sin mtdt =0, n = m h i 8 6 Z0 S is a basis of T [0, 2 ]. ⇒ π 12 Theorem 6.5 (Gram-Schmidt Process) Let u , u , , u be a basis for a subspace W of n.Define { 1 2 ··· k} R v1 = u1, u v v = u 2 · 1 v , 2 2 − v 2 1 || 1|| . uk v1 uk v2 uk vk 1 − vk = uk · 2 v1 · 2 v2 · 2 vk 1. − v1 − v2 −···− vk 1 − || || || || || − || Then v , v , , v is an orthogonal set of nonzero vectors such that { 1 2 ··· i} Span v , v , , v = Span u , u , , u { 1 2 ··· i} { 1 2 ··· i} for each i.So v , v , , v is an orthogonal basis for W . { 1 2 ··· k} 13 Gram-Schmidt Process for any inner product space Let u , u , , u be a basis for an inner product space V .Define { 1 2 ··· k} v1 = u1, u , v v = u h 2 1iv , 2 2 − v 2 1 || 1|| . uk, v1 uk, v2 uk, vk 1 vk = uk h iv1 h iv2 h − ivk 1. − v 2 − v 2 −···− v 2 − || 1|| || 1|| || 1|| Then v , v , , v is an orthogonal set of nonzero vectors such that { 1 2 ··· i} Span v , v , , v = Span u , u , , u { 1 2 ··· i} { 1 2 ··· i} for each i.So v , v , , v is an orthogonal basis for W . { 1 2 ··· k} Proposition: The Gram-Schmidt process is valid for any inner product space. Proof You show it. Corollary: Every finite-dimensional inner product space has an 14 orthonormal basis. Example: P2 is an inner product space with the following inner product 1 〈 f , g〉 = f (x)g(x)dx ∫−1 2 ∀ f , g ∈ P2. From a basis B = {1, x, x } = {u1, u2, u3} of P2, an orthogonal basis {v1, v2, v3} may be obtained by applying the Gram-Schmidt process. 15 Example: P2 is an inner product space with the following inner product 1 〈 f , g〉 = f (x)g(x)dx ∫−1 2 ∀ f , g ∈ P2. From a basis B = {1, x, x } = {u1, u2, u3} of P2, an orthogonal basis {v1, v2, v3} may be obtained by applying the Gram-Schmidt process. v1 = u1 =1 1 u2, v1 1 t 1dt − · v2 = u2 h 2iv1 = x 1 (1) = x 0 1=x − v1 − 2 − · R 1 1 dt || || − R 1 2 1 2 u3, v1 u3, v2 2 1 t 1dt 1 t tdt v3 = u3 h 2iv1 h 2iv2 = x − · (1) − · (x) − v1 − v2 − 1 2 − 1 2 || || || || R 1 1 dt R 1 t dt − − R R 2 1 = x2 3 1 0 x = x2 . − 2 · − · − 3 16 To get an orthonormal basis, compute 1 1 2 2 p 2 v1 = 1 dx = 2 v2 = x dx = || || s 1 || || s 1 3 Z− Z− r 1 2 2 1 8 v3 = x dx = || || s 1 − 3 45 Z− ✓ ◆ r and get 1 1 1 1 3 45 2 1 v1, v2, v3, = , x, x v1 v2 v3 p2 2 8 − 3 ⇢|| || || || || || ( r r ✓ ◆) 2 For P with the same inner product and the basis B = {1, x, x , }, the same procedure may be applied to obtain an orthonormal basis {p0(x), p1(x), p2(x), }, called the normalized Legendre polynomials.
Load more

Recommended publications
  • Math 317: Linear Algebra Notes and Problems
    Math 317: Linear Algebra Notes and Problems Nicholas Long SFASU Contents Introduction iv 1 Efficiently Solving Systems of Linear Equations and Matrix Op- erations 1 1.1 Warmup Problems . .1 1.2 Solving Linear Systems . .3 1.2.1 Geometric Interpretation of a Solution Set . .9 1.3 Vector and Matrix Equations . 11 1.4 Solution Sets of Linear Systems . 15 1.5 Applications . 18 1.6 Matrix operations . 20 1.6.1 Special Types of Matrices . 21 1.6.2 Matrix Multiplication . 22 2 Vector Spaces 25 2.1 Subspaces . 28 2.2 Span . 29 2.3 Linear Independence . 31 2.4 Linear Transformations . 33 2.5 Applications . 38 3 Connecting Ideas 40 3.1 Basis and Dimension . 40 3.1.1 rank and nullity . 41 3.1.2 Coordinate Vectors relative to a basis . 42 3.2 Invertible Matrices . 44 3.2.1 Elementary Matrices . 44 ii CONTENTS iii 3.2.2 Computing Inverses . 45 3.2.3 Invertible Matrix Theorem . 46 3.3 Invertible Linear Transformations . 47 3.4 LU factorization of matrices . 48 3.5 Determinants . 49 3.5.1 Computing Determinants . 49 3.5.2 Properties of Determinants . 51 3.6 Eigenvalues and Eigenvectors . 52 3.6.1 Diagonalizability . 55 3.6.2 Eigenvalues and Eigenvectors of Linear Transfor- mations . 56 4 Inner Product Spaces 58 4.1 Inner Products . 58 4.2 Orthogonal Complements . 59 4.3 QR Decompositions . 59 Nicholas Long Introduction In this course you will learn about linear algebra by solving a carefully designed sequence of problems. It is important that you understand every problem and proof.
    [Show full text]
  • Notes, Since the Quadratic Form Associated with an Inner Product (Kuk2 = Hu, Ui) Must Be Positive Def- Inite
    Bindel, Spring 2012 Intro to Scientific Computing (CS 3220) Week 3: Wednesday, Feb 8 Spaces and bases 1 n I have two favorite vector spaces : R and the space Pd of polynomials of degree at most d. For Rn, we have a canonical basis: n R = spanfe1; e2; : : : ; eng; where ek is the kth column of the identity matrix. This basis is frequently convenient both for analysis and for computation. For Pd, an obvious- seeming choice of basis is the power basis: 2 d Pd = spanf1; x; x ; : : : ; x g: But this obvious-looking choice turns out to often be terrible for computation. Why? The short version is that powers of x aren't all that strongly linearly dependent, but we need to develop some more concepts before that short description will make much sense. The range space of a matrix or a linear map A is just the set of vectors y that can be written in the form y = Ax. If A is full (column) rank, then the columns of A are linearly independent, and they form a basis for the range space. Otherwise, A is rank-deficient, and there is a non-trivial null space consisting of vectors x such that Ax = 0. Rank deficiency is a delicate property2. For example, consider the matrix 1 1 A = : 1 1 This matrix is rank deficient, but the matrix 1 + δ 1 A^ = : 1 1 is not rank deficient for any δ 6= 0. Technically, the columns of A^ form a basis for R2, but we should be disturbed by the fact that A^ is so close to a singular matrix.
    [Show full text]
  • Estimations of the Trace of Powers of Positive Self-Adjoint Operators by Extrapolation of the Moments∗
    Electronic Transactions on Numerical Analysis. ETNA Volume 39, pp. 144-155, 2012. Kent State University Copyright 2012, Kent State University. http://etna.math.kent.edu ISSN 1068-9613. ESTIMATIONS OF THE TRACE OF POWERS OF POSITIVE SELF-ADJOINT OPERATORS BY EXTRAPOLATION OF THE MOMENTS∗ CLAUDE BREZINSKI†, PARASKEVI FIKA‡, AND MARILENA MITROULI‡ Abstract. Let A be a positive self-adjoint linear operator on a real separable Hilbert space H. Our aim is to build estimates of the trace of Aq, for q ∈ R. These estimates are obtained by extrapolation of the moments of A. Applications of the matrix case are discussed, and numerical results are given. Key words. Trace, positive self-adjoint linear operator, symmetric matrix, matrix powers, matrix moments, extrapolation. AMS subject classifications. 65F15, 65F30, 65B05, 65C05, 65J10, 15A18, 15A45. 1. Introduction. Let A be a positive self-adjoint linear operator from H to H, where H is a real separable Hilbert space with inner product denoted by (·, ·). Our aim is to build estimates of the trace of Aq, for q ∈ R. These estimates are obtained by extrapolation of the integer moments (z, Anz) of A, for n ∈ N. A similar procedure was first introduced in [3] for estimating the Euclidean norm of the error when solving a system of linear equations, which corresponds to q = −2. The case q = −1, which leads to estimates of the trace of the inverse of a matrix, was studied in [4]; on this problem, see [10]. Let us mention that, when only positive powers of A are used, the Hilbert space H could be infinite dimensional, while, for negative powers of A, it is always assumed to be a finite dimensional one, and, obviously, A is also assumed to be invertible.
    [Show full text]
  • 5 the Dirac Equation and Spinors
    5 The Dirac Equation and Spinors In this section we develop the appropriate wavefunctions for fundamental fermions and bosons. 5.1 Notation Review The three dimension differential operator is : ∂ ∂ ∂ = , , (5.1) ∂x ∂y ∂z We can generalise this to four dimensions ∂µ: 1 ∂ ∂ ∂ ∂ ∂ = , , , (5.2) µ c ∂t ∂x ∂y ∂z 5.2 The Schr¨odinger Equation First consider a classical non-relativistic particle of mass m in a potential U. The energy-momentum relationship is: p2 E = + U (5.3) 2m we can substitute the differential operators: ∂ Eˆ i pˆ i (5.4) → ∂t →− to obtain the non-relativistic Schr¨odinger Equation (with = 1): ∂ψ 1 i = 2 + U ψ (5.5) ∂t −2m For U = 0, the free particle solutions are: iEt ψ(x, t) e− ψ(x) (5.6) ∝ and the probability density ρ and current j are given by: 2 i ρ = ψ(x) j = ψ∗ ψ ψ ψ∗ (5.7) | | −2m − with conservation of probability giving the continuity equation: ∂ρ + j =0, (5.8) ∂t · Or in Covariant notation: µ µ ∂µj = 0 with j =(ρ,j) (5.9) The Schr¨odinger equation is 1st order in ∂/∂t but second order in ∂/∂x. However, as we are going to be dealing with relativistic particles, space and time should be treated equally. 25 5.3 The Klein-Gordon Equation For a relativistic particle the energy-momentum relationship is: p p = p pµ = E2 p 2 = m2 (5.10) · µ − | | Substituting the equation (5.4), leads to the relativistic Klein-Gordon equation: ∂2 + 2 ψ = m2ψ (5.11) −∂t2 The free particle solutions are plane waves: ip x i(Et p x) ψ e− · = e− − · (5.12) ∝ The Klein-Gordon equation successfully describes spin 0 particles in relativistic quan- tum field theory.
    [Show full text]
  • A Some Basic Rules of Tensor Calculus
    A Some Basic Rules of Tensor Calculus The tensor calculus is a powerful tool for the description of the fundamentals in con- tinuum mechanics and the derivation of the governing equations for applied prob- lems. In general, there are two possibilities for the representation of the tensors and the tensorial equations: – the direct (symbolic) notation and – the index (component) notation The direct notation operates with scalars, vectors and tensors as physical objects defined in the three dimensional space. A vector (first rank tensor) a is considered as a directed line segment rather than a triple of numbers (coordinates). A second rank tensor A is any finite sum of ordered vector pairs A = a b + ... +c d. The scalars, vectors and tensors are handled as invariant (independent⊗ from the choice⊗ of the coordinate system) objects. This is the reason for the use of the direct notation in the modern literature of mechanics and rheology, e.g. [29, 32, 49, 123, 131, 199, 246, 313, 334] among others. The index notation deals with components or coordinates of vectors and tensors. For a selected basis, e.g. gi, i = 1, 2, 3 one can write a = aig , A = aibj + ... + cidj g g i i ⊗ j Here the Einstein’s summation convention is used: in one expression the twice re- peated indices are summed up from 1 to 3, e.g. 3 3 k k ik ik a gk ∑ a gk, A bk ∑ A bk ≡ k=1 ≡ k=1 In the above examples k is a so-called dummy index. Within the index notation the basic operations with tensors are defined with respect to their coordinates, e.
    [Show full text]
  • 18.700 JORDAN NORMAL FORM NOTES These Are Some Supplementary Notes on How to Find the Jordan Normal Form of a Small Matrix. Firs
    18.700 JORDAN NORMAL FORM NOTES These are some supplementary notes on how to find the Jordan normal form of a small matrix. First we recall some of the facts from lecture, next we give the general algorithm for finding the Jordan normal form of a linear operator, and then we will see how this works for small matrices. 1. Facts Throughout we will work over the field C of complex numbers, but if you like you may replace this with any other algebraically closed field. Suppose that V is a C-vector space of dimension n and suppose that T : V → V is a C-linear operator. Then the characteristic polynomial of T factors into a product of linear terms, and the irreducible factorization has the form m1 m2 mr cT (X) = (X − λ1) (X − λ2) ... (X − λr) , (1) for some distinct numbers λ1, . , λr ∈ C and with each mi an integer m1 ≥ 1 such that m1 + ··· + mr = n. Recall that for each eigenvalue λi, the eigenspace Eλi is the kernel of T − λiIV . We generalized this by defining for each integer k = 1, 2,... the vector subspace k k E(X−λi) = ker(T − λiIV ) . (2) It is clear that we have inclusions 2 e Eλi = EX−λi ⊂ E(X−λi) ⊂ · · · ⊂ E(X−λi) ⊂ .... (3) k k+1 Since dim(V ) = n, it cannot happen that each dim(E(X−λi) ) < dim(E(X−λi) ), for each e e +1 k = 1, . , n. Therefore there is some least integer ei ≤ n such that E(X−λi) i = E(X−λi) i .
    [Show full text]
  • Linear Algebra) 2
    MATH 680 Computation Intensive Statistics December 3, 2018 Matrix Basics Contents 1 Rank (linear algebra) 2 1.1 Proofs that column rank = row rank . .2 1.2 Properties . .4 2 Kernel (linear algebra) 5 2.1 Properties . .6 2.2 Illustration . .9 3 Matrix Norm 12 3.1 Matrix norms induced by vector norms . 12 3.2 Entrywise matrix norms . 14 3.3 Schatten norms . 18 1 1 RANK (LINEAR ALGEBRA) 2 4 Projection 19 4.1 Properties and classification . 20 4.2 Orthogonal projections . 21 1 Rank (linear algebra) In linear algebra, the rank of a matrix A is the dimension of the vector space generated (or spanned) by its columns. This corresponds to the maximal number of linearly independent columns of A. This, in turn, is identical to the dimension of the space spanned by its rows. The column rank of A is the dimension of the column space of A, while the row rank of A is the dimension of the row space of A. A fundamental result in linear algebra is that the column rank and the row rank are always equal. 1.1 Proofs that column rank = row rank Let A be an m × n matrix with entries in the real numbers whose row rank is r. There- fore, the dimension of the row space of A is r. Let x1; x2; : : : ; xr be a basis of the row space of A. We claim that the vectors Ax1; Ax2; : : : ; Axr are linearly independent. To see why, consider a linear homogeneous relation involving these vectors with scalar coefficients c1; c2; : : : ; cr : 0 = c1Ax1 + c2Ax2 + ··· + crAxr = A(c1x1 + c2x2 + ··· + crxr) = Av; 1 RANK (LINEAR ALGEBRA) 3 where v = c1x1 + c2x2 + ··· + crxr: We make two observations: 1.
    [Show full text]
  • Inner Products and Orthogonality
    Advanced Linear Algebra – Week 5 Inner Products and Orthogonality This week we will learn about: • Inner products (and the dot product again), • The norm induced by the inner product, • The Cauchy–Schwarz and triangle inequalities, and • Orthogonality. Extra reading and watching: • Sections 1.3.4 and 1.4.1 in the textbook • Lecture videos 17, 18, 19, 20, 21, and 22 on YouTube • Inner product space at Wikipedia • Cauchy–Schwarz inequality at Wikipedia • Gram–Schmidt process at Wikipedia Extra textbook problems: ? 1.3.3, 1.3.4, 1.4.1 ?? 1.3.9, 1.3.10, 1.3.12, 1.3.13, 1.4.2, 1.4.5(a,d) ??? 1.3.11, 1.3.14, 1.3.15, 1.3.25, 1.4.16 A 1.3.18 1 Advanced Linear Algebra – Week 5 2 There are many times when we would like to be able to talk about the angle between vectors in a vector space V, and in particular orthogonality of vectors, just like we did in Rn in the previous course. This requires us to have a generalization of the dot product to arbitrary vector spaces. Definition 5.1 — Inner Product Suppose that F = R or F = C, and V is a vector space over F. Then an inner product on V is a function h·, ·i : V × V → F such that the following three properties hold for all c ∈ F and all v, w, x ∈ V: a) hv, wi = hw, vi (conjugate symmetry) b) hv, w + cxi = hv, wi + chv, xi (linearity in 2nd entry) c) hv, vi ≥ 0, with equality if and only if v = 0.
    [Show full text]
  • Trace Inequalities for Matrices
    Bull. Aust. Math. Soc. 87 (2013), 139–148 doi:10.1017/S0004972712000627 TRACE INEQUALITIES FOR MATRICES KHALID SHEBRAWI ˛ and HUSSIEN ALBADAWI (Received 23 February 2012; accepted 20 June 2012) Abstract Trace inequalities for sums and products of matrices are presented. Relations between the given inequalities and earlier results are discussed. Among other inequalities, it is shown that if A and B are positive semidefinite matrices then tr(AB)k ≤ minfkAkk tr Bk; kBkk tr Akg for any positive integer k. 2010 Mathematics subject classification: primary 15A18; secondary 15A42, 15A45. Keywords and phrases: trace inequalities, eigenvalues, singular values. 1. Introduction Let Mn(C) be the algebra of all n × n matrices over the complex number field. The singular values of A 2 Mn(C), denoted by s1(A); s2(A);:::; sn(A), are the eigenvalues ∗ 1=2 of the matrix jAj = (A A) arranged in such a way that s1(A) ≥ s2(A) ≥ · · · ≥ sn(A). 2 ∗ ∗ Note that si (A) = λi(A A) = λi(AA ), so for a positive semidefinite matrix A, we have si(A) = λi(A)(i = 1; 2;:::; n). The trace functional of A 2 Mn(C), denoted by tr A or tr(A), is defined to be the sum of the entries on the main diagonal of A and it is well known that the trace of a Pn matrix A is equal to the sum of its eigenvalues, that is, tr A = j=1 λ j(A). Two principal properties of the trace are that it is a linear functional and, for A; B 2 Mn(C), we have tr(AB) = tr(BA).
    [Show full text]
  • Lecture 28: Eigenvalues Allowing Complex Eigenvalues Is Really a Blessing
    Math 19b: Linear Algebra with Probability Oliver Knill, Spring 2011 cos(t) sin(t) 2 2 For a rotation A = the characteristic polynomial is λ − 2 cos(α)+1 − sin(t) cos(t) which has the roots cos(α) ± i sin(α)= eiα. Lecture 28: Eigenvalues Allowing complex eigenvalues is really a blessing. The structure is very simple: Fundamental theorem of algebra: For a n × n matrix A, the characteristic We have seen that det(A) = 0 if and only if A is invertible. polynomial has exactly n roots. There are therefore exactly n eigenvalues of A if we count them with multiplicity. The polynomial fA(λ) = det(A − λIn) is called the characteristic polynomial of 1 n n−1 A. Proof One only has to show a polynomial p(z)= z + an−1z + ··· + a1z + a0 always has a root z0 We can then factor out p(z)=(z − z0)g(z) where g(z) is a polynomial of degree (n − 1) and The eigenvalues of A are the roots of the characteristic polynomial. use induction in n. Assume now that in contrary the polynomial p has no root. Cauchy’s integral theorem then tells dz 2πi Proof. If Av = λv,then v is in the kernel of A − λIn. Consequently, A − λIn is not invertible and = =0 . (1) z =r | | | zp(z) p(0) det(A − λIn)=0 . On the other hand, for all r, 2 1 dz 1 2π 1 For the matrix A = , the characteristic polynomial is | | ≤ 2πrmax|z|=r = . (2) z =r 4 −1 | | | zp(z) |zp(z)| min|z|=rp(z) The right hand side goes to 0 for r →∞ because 2 − λ 1 2 det(A − λI2) = det( )= λ − λ − 6 .
    [Show full text]
  • Approximating Spectral Sums of Large-Scale Matrices Using Stochastic Chebyshev Approximations∗
    Approximating Spectral Sums of Large-scale Matrices using Stochastic Chebyshev Approximations∗ Insu Han y Dmitry Malioutov z Haim Avron x Jinwoo Shin { March 10, 2017 Abstract Computation of the trace of a matrix function plays an important role in many scientific com- puting applications, including applications in machine learning, computational physics (e.g., lat- tice quantum chromodynamics), network analysis and computational biology (e.g., protein fold- ing), just to name a few application areas. We propose a linear-time randomized algorithm for approximating the trace of matrix functions of large symmetric matrices. Our algorithm is based on coupling function approximation using Chebyshev interpolation with stochastic trace estima- tors (Hutchinson’s method), and as such requires only implicit access to the matrix, in the form of a function that maps a vector to the product of the matrix and the vector. We provide rigorous approximation error in terms of the extremal eigenvalue of the input matrix, and the Bernstein ellipse that corresponds to the function at hand. Based on our general scheme, we provide algo- rithms with provable guarantees for important matrix computations, including log-determinant, trace of matrix inverse, Estrada index, Schatten p-norm, and testing positive definiteness. We experimentally evaluate our algorithm and demonstrate its effectiveness on matrices with tens of millions dimensions. 1 Introduction Given a symmetric matrix A 2 Rd×d and function f : R ! R, we study how to efficiently compute d X Σf (A) = tr(f(A)) = f(λi); (1) i=1 arXiv:1606.00942v2 [cs.DS] 9 Mar 2017 where λ1; : : : ; λd are eigenvalues of A.
    [Show full text]
  • Math 217: True False Practice Professor Karen Smith 1. a Square Matrix Is Invertible If and Only If Zero Is Not an Eigenvalue. Solution Note: True
    (c)2015 UM Math Dept licensed under a Creative Commons By-NC-SA 4.0 International License. Math 217: True False Practice Professor Karen Smith 1. A square matrix is invertible if and only if zero is not an eigenvalue. Solution note: True. Zero is an eigenvalue means that there is a non-zero element in the kernel. For a square matrix, being invertible is the same as having kernel zero. 2. If A and B are 2 × 2 matrices, both with eigenvalue 5, then AB also has eigenvalue 5. Solution note: False. This is silly. Let A = B = 5I2. Then the eigenvalues of AB are 25. 3. If A and B are 2 × 2 matrices, both with eigenvalue 5, then A + B also has eigenvalue 5. Solution note: False. This is silly. Let A = B = 5I2. Then the eigenvalues of A + B are 10. 4. A square matrix has determinant zero if and only if zero is an eigenvalue. Solution note: True. Both conditions are the same as the kernel being non-zero. 5. If B is the B-matrix of some linear transformation V !T V . Then for all ~v 2 V , we have B[~v]B = [T (~v)]B. Solution note: True. This is the definition of B-matrix. 21 2 33 T 6. Suppose 40 2 05 is the matrix of a transformation V ! V with respect to some basis 0 0 1 B = (f1; f2; f3). Then f1 is an eigenvector. Solution note: True. It has eigenvalue 1. The first column of the B-matrix is telling us that T (f1) = f1.
    [Show full text]