DOCSLIB.ORG

Spectrum and Pseudospectrum of a Tensor

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Spectrum and Pseudospectrum of a Tensor Spectrum and Pseudospectrum of a Tensor Lek-Heng Lim University of California, Berkeley July 11, 2008 L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 1 / 27 Matrix eigenvalues and eigenvectors One of the most important ideas ever invented. I R. Coifman et. al.: \Eigenvector magic: eigenvectors as an extension of Newtonian calculus." Normal/Hermitian A I Invariant subspace: Ax = λx. > > I Rayleigh quotient: x Ax=x x. > 2 I Lagrange multipliers: x Ax − λ(kxk − 1). > I Best rank-1 approximation: minkxk=1kA − λxx k. Nonnormal A −1 −1 I Pseudospectrum: σ"(A) = fλ 2 C j k(A − λI ) k > " g. ∗ I Numerical range: W (A) = fx Ax 2 C j kxk = 1g. ∗ I Irreducible representations of C (A) with natural Borel structure. ∗ I Primitive ideals of C (A) with hull-kernel topology. How can one define these for tensors? L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 2 / 27 DARPA mathematical challenge eight One of the twenty three mathematical challenges announced at DARPA Tech 2007. Problem Beyond convex optimization: can linear algebra be replaced by algebraic geometry in a systematic way? Algebraic geometry in a slogan: polynomials are to algebraic geometry what matrices are to linear algebra. Polynomial f 2 R[x1;:::; xn] of degree d can be expressed as > > f (x) = a0 + a1 x + x A2x + A3(x; x; x) + ··· + Ad (x;:::; x): n n×n n×n×n n×···×n a0 2 R; a1 2 R ; A2 2 R ; A3 2 R ;:::; Ad 2 R . Numerical linear algebra: d = 2. Numerical multilinear algebra: d > 2. L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 3 / 27 Hypermatrices Totally ordered finite sets: [n] = f1 < 2 < ··· < ng, n 2 N. Vector or n-tuple f :[n] ! R: > n If f (i) = ai , then f is represented by a = [a1;:::; an] 2 R . Matrix f :[m] × [n] ! R: m;n m×n If f (i; j) = aij , then f is represented by A = [aij ]i;j=1 2 R . Hypermatrix (order 3) f :[l] × [m] × [n] ! R: l;m;n l×m×n If f (i; j; k) = aijk , then f is represented by A = aijk i;j;k=1 2 R . J K X [n] [m]×[n] [l]×[m]×[n] Normally R = ff : X ! Rg. Ought to be R ; R ; R . L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 4 / 27 Hypermatrices and tensors Up to choice of bases n a 2 R can represent a vector in V (contravariant) or a linear functional in V ∗ (covariant). m×n A 2 R can represent a bilinear form V × W ! R (contravariant), ∗ ∗ a bilinear form V × W ! R (covariant), or a linear operator V ! W (mixed). l×m×n A 2 R can represent trilinear form U × V × W ! R (contravariant), bilinear operators V × W ! U (mixed), etc. A hypermatrix is the same as a tensor if 1 we give it coordinates (represent with respect to some bases); 2 we ignore covariance and contravariance. L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 5 / 27 Basic operation on a hypermatrix m×n A matrix can be multiplied on the left and right: A 2 R , p×m q×n X 2 R , Y 2 R , > p×q (X ; Y ) · A = XAY = [cαβ] 2 R where Xm;n cαβ = xαi yβj aij : i;j=1 l×m×n A hypermatrix can be multiplied on three sides: A = aijk 2 R , p×l q×m r×n J K X 2 R , Y 2 R , Z 2 R , p×q×r (X ; Y ; Z) ·A = cαβγ 2 R J K where Xl;m;n cαβγ = xαi yβj zγk aijk : i;j;k=1 L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 6 / 27 Numerical range of a tensor Covariant version: A· (X >; Y >; Z >) := (X ; Y ; Z) ·A: Gives convenient notations for multilinear functionals and multilinear l m n operators. For x 2 R ; y 2 R ; z 2 R , Xl;m;n A(x; y; z) := A· (x; y; z) = aijk xi yj zk ; i;j;k=1 Xm;n A(I ; y; z) := A· (I ; y; z) = aijk yj zk : j;k=1 n×n Numerical range of square matrix A 2 C , ∗ ∗ W (A) = fx Ax 2 C j kxk2 = 1g = fA(x; x ) 2 C j kxk2 = 1g: n×···×n Plausible generalization to cubical hypermatrix A 2 C , ( fA(x; x∗;:::; x) 2 j kxk = 1g odd order; W (A) = C k ∗ ∗ fA(x; x ;:::; x ) 2 C j kxkk = 1g even order: L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 7 / 27 Symmetric hypermatrices n×n×n Cubical hypermatrix aijk 2 R is symmetric if J K aijk = aikj = ajik = ajki = akij = akji : Invariant under all permutations σ 2 Sk on indices. k n S (R ) denotes set of all order-k symmetric hypermatrices. Example Higher order derivatives of multivariate functions. Example Moments of a random vector x = (X1;:::; Xn): Z Z n n mk (x) = E(xi1 xi2 ··· xik ) = ··· xi1 xi2 ··· xik dµ(xi1 ) ··· dµ(xik ) : i1;:::;ik =1 i1;:::;ik =1 L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 8 / 27 Symmetric hypermatrices Example Cumulants of a random vector x = (X1;:::; Xn): 2 3n X p−1 Q Q κk (x) = 4 (−1) (p − 1)!E xi ··· E xi 5 : i2A1 i2Ap A1t···tAp =fi1;:::;ik g i1;:::;ik =1 For n = 1, κk (x) for k = 1; 2; 3; 4 are the expectation, variance, skewness, and kurtosis. Important in Independent Component Analysis (ICA). L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 9 / 27 Multilinear spectral theory n×n×n Let A 2 R (easier if A symmetric). 1 How should one define its eigenvalues and eigenvectors? 2 What is a decomposition that generalizes the eigenvalue decomposition of a matrix? l×m×n Let A 2 R 1 How should one define its singualr values and singular vectors? 2 What is a decomposition that generalizes the singular value decomposition of a matrix? Somewhat surprising: (1) and (2) have different answers. L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 10 / 27 Tensor ranks (Hitchcock, 1927) m×n Matrix rank. A 2 R . rank(A) = dim(span fA ;:::; A g) (column rank) R •1 •n = dim(span fA ;:::; A g) (row rank) R 1• m• Pr T = minfr j A = i=1ui vi g (outer product rank): Multilinear rank. A 2 l×m×n. rank (A) = (r (A); r (A); r (A)), R 1 2 3 r (A) = dim(span fA ;:::; A g) 1 R 1•• l•• r (A) = dim(span fA ;:::; A g) 2 R •1• •m• r (A) = dim(span fA ;:::; A g) 3 R ••1 ••n l×m×n Outer product rank. A 2 R . Pr rank⊗(A) = minfr j A = i=1ui ⊗ vi ⊗ wi g l;m;n where u ⊗ v ⊗ w : = ui vj wk i;j;k=1. J K L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 11 / 27 Matrix EVD and SVD Rank revealing decompositions. 2 n Symmetric eigenvalue decomposition of A 2 S (R ), > Xr A = V ΛV = λi vi ⊗ vi ; i=1 where rank(A) = r, V 2 O(n) eigenvectors, Λ eigenvalues. m×n Singular value decomposition of A 2 R , > Xr A = UΣV = σi ui ⊗ vi i=1 where rank(A) = r, U 2 O(m) left singular vectors, V 2 O(n) right singular vectors, Σ singular values. L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 12 / 27 One plausible EVD and SVD for hypermatrices Rank revealing decompositions associated with the outer product rank. 3 n Symmetric outer product decomposition of A 2 S (R ), Xr A = λi vi ⊗ vi ⊗ vi i=1 where rankS(A) = r, vi unit vector, λi 2 R. l×m×n Outer product decomposition of A 2 R , Xr A = σi ui ⊗ vi ⊗ wi i=1 l m n where rank⊗(A) = r, ui 2 R ; vi 2 R ; wi 2 R unit vectors, σi 2 R. L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 13 / 27 Another plausible EVD and SVD for hypermatrices Rank revealing decompositions associated with the multilinear rank. l×m×n Singular value decomposition of A 2 R , A = (U; V ; W ) ·C where rank (A) = (r ; r ; r ), U 2 l×r1 , V 2 m×r2 , W 2 n×r3 1 2 3 R R R r ×r ×r have orthonormal columns and C 2 R 1 2 3 . 3 n Symmetric eigenvalue decomposition of A 2 S (R ), A = (U; U; U) ·C where rank (A) = (r; r; r), U 2 n×r has orthonormal columns and R 3 r C 2 S (R ). L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 14 / 27 Variational approach to eigenvalues/vectors m×n A 2 R symmetric. Eigenvalues and eigenvectors are critical values and critical points of > 2 x Ax=kxk2: Equivalently, critical values/points of x>Ax constrained to unit sphere. Lagrangian: > 2 L(x; λ) = x Ax − λ(kxk2 − 1): n Vanishing of rL at critical (xc ; λc ) 2 R × R yields familiar Axc = λc xc : L.-H. Lim (Berkeley) Spectrum and Pseudospectrum of a Tensor July 11, 2008 15 / 27 Variational approach to singular values/vectors m×n A 2 R .
Load more

Recommended publications
  • Criss-Cross Type Algorithms for Computing the Real Pseudospectral Abscissa
    CRISS-CROSS TYPE ALGORITHMS FOR COMPUTING THE REAL PSEUDOSPECTRAL ABSCISSA DING LU∗ AND BART VANDEREYCKEN∗ Abstract. The real "-pseudospectrum of a real matrix A consists of the eigenvalues of all real matrices that are "-close to A. The closeness is commonly measured in spectral or Frobenius norm. The real "-pseudospectral abscissa, which is the largest real part of these eigenvalues for a prescribed value ", measures the structured robust stability of A. In this paper, we introduce a criss-cross type algorithm to compute the real "-pseudospectral abscissa for the spectral norm. Our algorithm is based on a superset characterization of the real pseudospectrum where each criss and cross search involves solving linear eigenvalue problems and singular value optimization problems. The new algorithm is proved to be globally convergent, and observed to be locally linearly convergent. Moreover, we propose a subspace projection framework in which we combine the criss-cross algorithm with subspace projection techniques to solve large-scale problems. The subspace acceleration is proved to be locally superlinearly convergent. The robustness and efficiency of the proposed algorithms are demonstrated on numerical examples. Key words. eigenvalue problem, real pseudospectra, spectral abscissa, subspace methods, criss- cross methods, robust stability AMS subject classifications. 15A18, 93B35, 30E10, 65F15 1. Introduction. The real "-pseudospectrum of a matrix A 2 Rn×n is defined as R n×n (1) Λ" (A) = fλ 2 C : λ 2 Λ(A + E) with E 2 R ; kEk ≤ "g; where Λ(A) denotes the eigenvalues of a matrix A and k · k is the spectral norm. It is also known as the real unstructured spectral value set (see, e.g., [9, 13, 11]) and it can be regarded as a generalization of the more standard complex pseudospectrum (see, e.g., [23, 24]).
    [Show full text]
  • Reproducing Kernel Hilbert Space Compactification of Unitary Evolution
    Reproducing kernel Hilbert space compactification of unitary evolution groups Suddhasattwa Dasa, Dimitrios Giannakisa,∗, Joanna Slawinskab aCourant Institute of Mathematical Sciences, New York University, New York, NY 10012, USA bFinnish Center for Artificial Intelligence, Department of Computer Science, University of Helsinki, Helsinki, Finland Abstract A framework for coherent pattern extraction and prediction of observables of measure-preserving, ergodic dynamical systems with both atomic and continuous spectral components is developed. This framework is based on an approximation of the generator of the system by a compact operator Wτ on a reproducing kernel Hilbert space (RKHS). A key element of this approach is that Wτ is skew-adjoint (unlike regularization approaches based on the addition of diffusion), and thus can be characterized by a unique projection- valued measure, discrete by compactness, and an associated orthonormal basis of eigenfunctions. These eigenfunctions can be ordered in terms of a Dirichlet energy on the RKHS, and provide a notion of coherent observables under the dynamics akin to the Koopman eigenfunctions associated with the atomic part of the spectrum. In addition, the regularized generator has a well-defined Borel functional calculus allowing tWτ the construction of a unitary evolution group fe gt2R on the RKHS, which approximates the unitary Koopman evolution group of the original system. We establish convergence results for the spectrum and Borel functional calculus of the regularized generator to those of the original system in the limit τ ! 0+. Convergence results are also established for a data-driven formulation, where these operators are approximated using finite-rank operators obtained from observed time series. An advantage of working in spaces of observables with an RKHS structure is that one can perform pointwise evaluation and interpolation through bounded linear operators, which is not possible in Lp spaces.
    [Show full text]
  • Banach J. Math. Anal. 6 (2012), No. 1, 45–60 LINEAR MAPS
    Banach J. Math. Anal. 6 (2012), no. 1, 45–60 Banach Journal of Mathematical Analysis ISSN: 1735-8787 (electronic) www.emis.de/journals/BJMA/ LINEAR MAPS PRESERVING PSEUDOSPECTRUM AND CONDITION SPECTRUM G. KRISHNA KUMAR1 AND S. H. KULKARNI2 Communicated by K. Jarosz Abstract. We discuss properties of pseudospectrum and condition spectrum of an element in a complex unital Banach algebra and its -perturbation. Sev- eral results are proved about linear maps preserving pseudospectrum/ condition spectrum. These include the following: (1) Let A, B be complex unital Banach algebras and > 0. Let Φ : A → B be an -pseudospectrum preserving linear onto map. Then Φ preserves spectrum. If A and B are uniform algebras, then, Φ is an isometric isomorphism. (2) Let A, B be uniform algebras and 0 < < 1. Let Φ : A → B be 0 an -condition spectrum preserving linear map. Then Φ is an -almost 0 multiplicative map, where , tend to zero simultaneously. 1. Introduction Let A be a complex Banach algebra with unit 1. We shall identify λ.1 with λ. We recall that the spectrum of an element a ∈ A is defined as −1 σ(a) = λ ∈ C : λ − a∈ / A , where A−1 is the set of all invertible elements of A. The spectral radius of an element a is defined as r(a) = sup{|λ| : λ ∈ σ(a)}. Date: Received: 20June 2011; Revised: 25 August 2011; Accepted: 8 September 2011. ∗ Corresponding author. 2010 Mathematics Subject Classification. Primary 47B49; Secondary 46H05, 46J05, 47S48. Key words and phrases. Pseudospectrum, condition spectrum, almost multiplicative map, linear preserver, perturbation. 45 46 G.
    [Show full text]
  • Lecture Notes on Spectra and Pseudospectra of Matrices and Operators
    Lecture Notes on Spectra and Pseudospectra of Matrices and Operators Arne Jensen Department of Mathematical Sciences Aalborg University c 2009 Abstract We give a short introduction to the pseudospectra of matrices and operators. We also review a number of results concerning matrices and bounded linear operators on a Hilbert space, and in particular results related to spectra. A few applications of the results are discussed. Contents 1 Introduction 2 2 Results from linear algebra 2 3 Some matrix results. Similarity transforms 7 4 Results from operator theory 10 5 Pseudospectra 16 6 Examples I 20 7 Perturbation Theory 27 8 Applications of pseudospectra I 34 9 Applications of pseudospectra II 41 10 Examples II 43 1 11 Some infinite dimensional examples 54 1 Introduction We give an introduction to the pseudospectra of matrices and operators, and give a few applications. Since these notes are intended for a wide audience, some elementary concepts are reviewed. We also note that one can understand the main points concerning pseudospectra already in the finite dimensional case. So the reader not familiar with operators on a separable Hilbert space can assume that the space is finite dimensional. Let us briefly outline the contents of these lecture notes. In Section 2 we recall some results from linear algebra, mainly to fix notation, and to recall some results that may not be included in standard courses on linear algebra. In Section 4 we state some results from the theory of bounded operators on a Hilbert space. We have decided to limit the exposition to the case of bounded operators.
    [Show full text]
  • Pseudospectrum of an Element of a Banach Algebra
    Tusi Advances in Operator Theory Mathematical Research https://doi.org/10.1007/s43036-019-00016-x Group ORIGINAL PAPER Decomposition of the (n, )-pseudospectrum of an element of a Banach algebra Kousik Dhara1 · S. H. Kulkarni2 Received: 20 June 2019 / Accepted: 9 October 2019 © Tusi Mathematical Research Group (TMRG) 2019 Abstract Let A be a complex Banach algebra with unit. For an integer n ≥ 0 and >0, the (n,)-pseudospectrum of a ∈ A is defined by −2n 1/2n 1 Λ ,(A, a) := λ ∈ C : (λ − a) A (λ − a) ≥ . n is not invertible in or Let p ∈ A be a nontrivial idempotent. Then pAp ={pbp : b ∈ A} is a Banach subalgebra of A with unit p, known as a reduced Banach algebra. Suppose ap = pa. We study the relationship of Λn,(A, a) and Λn,(pAp, pa). We extend this by considering first a finite family, and then an at most countable family of idempotents satisfying some conditions. We establish that under suitable assumptions, the (n,)- pseudospectrum of a can be decomposed into the union of the (n,)-pseudospectra of some elements in reduced Banach algebras. Keywords Banach algebra · Spectrum · Pseudospectrum · (n,)-Pseudospectrum Mathematics Subject Classification 47A10 · 46H05 · 47A12 Communicated by Chi-Keung Ng. B Kousik Dhara [email protected] S. H. Kulkarni [email protected] 1 Theoretical Statistics and Mathematics Unit, Indian Statistical Institute, Bangalore Centre, 8th Mile, Mysore Road, Bangalore 560059, India 2 Department of Mathematics, Indian Institute of Technology Palakkad, Palakkad, Kerala 678557, India K. Dhara and S. H. Kulkarni 1 Introduction It is known that the spectrum of the direct sum of a finite number of operators on the direct sum of Hilbert spaces is the union of their spectra (Problem 98 of [9]).
    [Show full text]
  • International Conference on Linear Algebra and Its Applications REPORT
    International Conference on Linear Algebra and its Applications December 11-15, 2017 REPORT The Fourth DAE-BRNS Theme Meeting on Generation and Use of Covariance Matrices in the Applications of Nuclear Data December 09-13, 2017 Department of Statistics, PSPH Level VI, Health Sciences Library Building Manipal Academy of Higher Education Manipal–576104, Karnataka, INDIA Editor Shreemathi S. Mayya Compiler David Raj Micheal January 15, 2018 MAHE honours save nature policy and limited number of this report are printed for internal circulation and for the communications with sponsors. Table of Contents 1 Overview of ICLAA 2017 & Theme Meeting on Covariance Matrix 5 1.1 Invited Delegates: ICLAA 20176 1.2 Delegates Contributing Paper: ICLAA 20178 1.3 Delegates Presenting Poster: ICLAA 2017 10 1.4 Fourth DAE-BRNS Theme Meeting on Covariance Matrix 10 1.5 Speakers in DAE-BRNS Theme Meeting 10 1.6 Acknowledgements 12 1.7 Special Issues 12 2 Committees 15 3 Message 19 4 From the Desk of Chairman (DAE-BRNS Theme Meeting) 21 5 Technical Committee: DAE-BRNS Theme Meeting 23 6 Program: DAE-BRNS Theme Meeting 25 7 Abstracts: DAE-BRNS Theme Meeting 29 8 List of Delegates: Theme Meeting 51 9 Program: ICLAA 2017 55 10 Abstracts: ICLAA 2017 61 10.1 Special Lectures & Plenary Talks 61 10.2 Invited Talks 67 10.3 Contributory Talks 85 10.4 Posters 131 11 List of Delegates: ICLAA 2017 137 3 Overview of ICLAA 2017 & Theme Meeting on Covariance Matrix International conference on Linear Al- gebra and its Applications–ICLAA 2017, third in its sequence, following CMT- GIM 2012 and ICLAA 2014, held in Manipal Academy of Higher Educa- tion, Manipal, India in December 11-15, 2017.
    [Show full text]
  • Mathematical Analysis of Quantum Mechanics with Non-Self-Adjoint Operators Radek Novak
    Mathematical analysis of quantum mechanics with non-self-adjoint operators Radek Novak To cite this version: Radek Novak. Mathematical analysis of quantum mechanics with non-self-adjoint operators. Func- tional Analysis [math.FA]. Université de Nantes; Czech Technical University in Prague, 2018. English. tel-01903520 HAL Id: tel-01903520 https://hal.archives-ouvertes.fr/tel-01903520 Submitted on 24 Oct 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. THESE DE DOCTORAT DE L'UNIVERSITE DE NANTES ET L’UNIVERSITE TECHNIQUE DE PRAGUE COMUE UNIVERSITE BRETAGNE LOIRE ECOLE DOCTORALE N° 601 Mathématiques et Sciences et Technologies de l'Information et de la Communication Spécialité : Mathématiques et leurs Interactions Par Radek NOVÁK Mathematical analysis of quantum mechanics with non-self-adjoint operators Thèse présentée et soutenue à Nantes, le 19 octobre 2018 Unité de recherche : Laboratoire de Mathématiques Jean Leray Rapporteurs avant soutenance : Michael LEVITIN Professeur, Université de Reading Thierry RAMOND Maître de Conférences, Université
    [Show full text]
  • ON the OPEN BALL CENTERED at an INVERTIBLE ELEMENT of a BANACH ALGEBRA 1. Introduction Let a Be a Complex Unital Banach Algebra
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Research Archive of Indian Institute of Technology Hyderabad O perators a nd & M atrices www.ele-math.com ON THE OPEN BALL CENTERED AT AN INVERTIBLE ELEMENT OF A BANACH ALGEBRA GEETHIKA SEBASTIAN AND SUKUMAR DANIEL Submitted to Operators and Matrices Abstract. Let A be a complex unital Banach algebra. Since the set of invertible elements is open, there is an open ball around every invertible element. In this article, we investigate the Banach algebras for which the radius given by the Neumann series is optimal. 1. Introduction Let A be a complex unital Banach algebra with unit e. G(A) and Sing(A) denote the set of invertible and singular elements of A respectively. The spectrum, spectral radius and the resolvent of an element a in A, are denoted by s(a), r(a) and r(a) respectively. It is well known that G(A) is open in A, as the open ball centered at any invert- a 1 B a; 1 G(A) ible element with radius ka−1k , denoted by ka−1k , is contained in ([2] Theorem 2.11). It is a natural to ask if there is a bigger open ball centered at a inside G(A)? It will be convenient to have the following definition. DEFINITION 1. An element a 2 G(A) is said to satisfy condition (B) (or belongs to the B class) if 1 B a; \ Sing(A) 6= f: ka−1k We say a Banach algebra A satisfies condition (B) if every a 2 G(A) satisfies condition (B).
    [Show full text]
  • Minimizing Communication for Eigenproblems and the Singular Value Decomposition
    Minimizing Communication for Eigenproblems and the Singular Value Decomposition Grey Ballard∗, James Demmel†, and Ioana Dumitriu‡ November 12, 2010 Abstract Algorithms have two costs: arithmetic and communication. The latter represents the cost of moving data, either between levels of a memory hierarchy, or between processors over a network. Communication often dominates arithmetic and represents a rapidly increasing proportion of the total cost, so we seek algorithms that minimize communication. In [4] lower bounds were presented on the amount of communi- cation required for essentially all O(n3)-like algorithms for linear algebra, including eigenvalue problems and the SVD. Conventional algorithms, including those currently implemented in (Sca)LAPACK, perform asymptotically more communication than these lower bounds require. In this paper we present parallel and sequential eigenvalue algorithms (for pencils, nonsymmetric matrices, and symmetric matrices) and SVD algorithms that do attain these lower bounds, and analyze their convergence and communication costs. 1 Introduction The running time of an algorithm depends on two factors: the number of floating point operations executed (arithmetic) and the amount of data moved either between levels of a memory hierarchy in the case of sequential computing, or over a network connecting processors in the case of parallel computing (commu- nication). The simplest metric of communication is to count the total number of words moved (also called the bandwidth cost). Another simple metric is to count the number of messages containing these words (also known as the latency cost). On current hardware the cost of moving a single word, or that of sending a single message, already greatly exceed the cost of one arithmetic operation, and technology trends indicate that this processor-memory gap is growing exponentially over time.
    [Show full text]
  • And Singular Values)
    Finite-dimensional spectral theory part II: understanding the spectrum (and singular values) Ed Bueler MATH 617 Functional Analysis Spring 2020 Ed Bueler (MATH 617) Finite-dimensional spectral theory II Spring 2020 1 / 41 Outline 1 introduction 2 functional calculus 3 resolvents 4 orthogonal projectors 5 singular value decomposition 6 conclusion Ed Bueler (MATH 617) Finite-dimensional spectral theory II Spring 2020 2 / 41 what happened in part I see part I first: bueler.github.io/M617S20/slides1.pdf definition. for a square matrix A 2 Cn×n, the spectrum is the set σ(A) = λ 2 C Av = λv for some v 6= 0 we proved: ∗ n×n A = QTQ Schur decomposition for any A 2 C A = QΛQ∗ spectral theorem for normal (AA∗ = A∗A) matrices where Q is unitary, T is upper-triangular, and Λ is diagonal ◦ both decompositions “reveal” the spectrum: σ(A) = fdiagonal entries of T or Λg ◦ spectral theorem for hermitian matrices is sometimes called the principal axis decomposition for quadratic forms Ed Bueler (MATH 617) Finite-dimensional spectral theory II Spring 2020 3 / 41 goal for MATH 617 goal extend the spectral theorem to 1-dimensions only possible for linear operators on Hilbert spaces H ◦ inner product needed for adjoints and unitaries ◦ unitary maps needed because they preserve vector space and metric and adjoint structures textbook (Muscat) extends to compact normal operators on H ◦ the spectrum is eigenvalues (almost exclusively) recommended text (B. Hall, Quantum Theory for Mathematicians) extends further to bounded (continuous) normal operators on H ◦ spectrum is not only eigenvalues ◦ statement of theorem uses projector-valued measures Hall also extends to unbounded normal operators on H ◦ but we won’t get there .
    [Show full text]
  • A Characterization of S-Condition Pseudospectrum of Multivalued
    Filomat 33:15 (2019), 5037–5050 Published by Faculty of Sciences and Mathematics, https://doi.org/10.2298/FIL1915037A University of Nis,ˇ Serbia Available at: http://www.pmf.ni.ac.rs/filomat A Characterization of -Condition Pseudospectrum of Multivalued S Linear Operators Aymen Ammara, Aref Jeribia, Mayssa Zayeda aDepartment of Mathematics, Faculty of Sciences of Sfax, University of Sfax, soukra Road Km 3.5 B.P. 1171, 3000, Sfax, Tunisia Abstract. The main goal of this paper, is to extend the concept of condition pseudospectrum to the case of closed linear relations pencils and to reveal some of their properties. We will start by giving the definition and then we will focus on the characterization and some results concerning these pseudospectra. 1. Introduction The linear relation made their appearance in functional analysis motivated by the need to consider adjoints of non-densely defined linear differential operators and also through the need of considering the inverses of certain operators used in the study of some Cauchy problems associated with parabolic type equation in Banach spaces. The theory of linear relation is the one of the most exciting and influential field of research in modem mathematics. Applications of this theory can be found in economic theory, non cooperative games, artificial intelligence, medicine and existence of solutions for differential inclusions, we recall as examples ([1], [2], [5], [6], [13]). Several mathematical and physical problems lead to operators pencils, λ T (operator-valued functions of a complex argument). Recently the spectral theory of operator pencilsS− attracts an attention of many mathematicians. Mainly, the completeness of the root vectors and asymptotic distributions of characteristic values are considered.
    [Show full text]
  • The Spectrum and Pseudospectrum of Non-Self Adjoint Pseudodifferential
    Pure and Applied Mathematics Quarterly Volume 6, Number 3 (Special Issue: In honor of Joseph J. Kohn, Part 1 of 2 ) 815|827, 2010 The Spectrum and Pseudospectrum of Non-self Adjoint Pseudodi®erential Operators Charles L. Epstein Dedicated to Professor Joe Kohn on the occasion of his 75th birthday Abstract: We study the spectrum of relatively compact, non-self adjoint perturbations of self adjoint operators, with the formal properties of pseudo- di®erential operators. When the self adjoint operator is elliptic, we obtain an estimate for the location of the spectrum in complex plane, which provides a quantitative improvement of classical results of Keldy¸s,Gohberg and Krein. Keywords: spectrum, non-self adjoint operator, pseudodi®erential opera- tor, resolvent kernel Introduction If A : X ! Y is a closed operator from a dense domain D(A) ½ X; to another Banach space Y; then the resolvent set, the complement of the spectrum, is the ¡1 set of points ¸ 2 C; such that A ¡ ¸ has a bounded inverse, RA(¸) = (A ¡ ¸) : The operator RA(¸) is called the resolvent of A: We denote the resolvent set by ½(A) and its complement, the spectrum of A; by ¤(A): If X = Y is a Hilbert space, with inner product h¢; ¢i; then we say that A is self adjoint if (1) hAx; yi = hx; Ayi for all x; y 2 D(A); and the linear functional `(x) = hAx; yi is bounded only if y 2 D(A): In this case it is easy to see that the spectrum lies in R: The norm of the resolvent operator Received January 30, 2008 Research partially supported by NSF grant DMS06-03973 and by DARPA HR00110510057.
    [Show full text]