DOCSLIB.ORG

Iterative Methods in Linear Algebra (Part 2)

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Iterative Methods in Linear Algebra (Part 2) Iterative Methods in Linear Algebra (part 2) Stan Tomov Innovative Computing Laboratory Computer Science Department The University of Tennessee Wednesday April 23, 2008 CS 594-0004, 04-23-2008 CS 594-0004, 04-23-2008 Outline Part I Krylov iterative solvers Part II Convergence and preconditioning Part III Iterative eigen-solvers CS 594-0004, 04-23-2008 Part I Krylov iterative solvers CS 594-0004, 04-23-2008 Krylov iterative solvers Building blocks for Krylov iterative solvers covered so far Projection/minimization in a subspace Petrov-Galerkin conditions Least squares minimization, etc. Orthogonalization CGS and MGS Cholesky or Householder based QR CS 594-0004, 04-23-2008 Krylov iterative solvers We also covered abstract formulations for iterative solvers and eigen-solvers What are the goals of this lecture? Give specific examples of Krylov solvers Show how examples relate to the abstract formulation Show how examples relate to the building blocks covered so far, specidicly to Projection, and Orthogonalization But we are not going into the details! CS 594-0004, 04-23-2008 Krylov iterative solvers How are these techniques related to Krylov iterative solvers? Remember projection slides 26 & 27, Lecture 7 (left) Projection in a subspace is the basis for an iterative method Here projection is in V In Krylov methods V is the Krylov subspace 2 m−1 Km(A, r0) = span{r0, Ar0, A r0,..., A r0} where r0 ≡ b − Ax0 and x0 is an initial guess. Often V or W are orthonormalized The projection is ’easier’ to find when we work with an orthonormal basis (e.g. problem 4 from homework 5: projection in general vs orthonormal basis) The orthonormalization can be CGS, MGS, Cholesky or Householder based, etc. CS 594-0004, 04-23-2008 Krylov Iterative Methods To summarize, Krylov iterative methods in general expend the Krylov subspace by a matrix-vector product, and do a projection in it. Various methods result by specific choices of the expansion and projection. CS 594-0004, 04-23-2008 Krylov Iterative Methods A specific example with the Conjugate Gradient Method (CG) CS 594-0004, 04-23-2008 Conjugate Gradient Method The method is for SPD matrices Both V and W are the Krylov subspaces, i.e. at iteration i i−1 V ≡ W ≡ Ki (A, r0) ≡ span{r0, Ar0,..., A r0} The projection xi ∈ Ki (A, r0) satisfies the Petrov-Galerkin conditions (Axi , φ) = (b, φ), for ∀φ ∈ Ki (A, r0) CS 594-0004, 04-23-2008 Conjugate Gradient Method (continued) At every iteration there is a way (to be shown later) to construct a new search direction pi such that span{p0, p1,..., pi } ≡ Ki+1(A, r0) and (Api , pj ) = 0 for i 6= j. Note: A is SPD ⇒ (Api , pj ) ≡ (pi , pj )A can be used as an inner product, i.e. p0,..., pi is an (·, ·)A orthogonal basis for Ki+1(A, r0) ⇒ we can easily find xi+1 ≈ x as xi+1 = x0 + α0p0 + ··· + αi pi s.t. (Axi+1, pj ) = (b, pj ) for j = 0,..., i Namely, because of the (·, ·)A orthogonality of p0,..., pi at iteration i + 1 we have to find only αi (ri , pi ) (Axi+1, pj ) = (A(xi + αi pi ), pi ) = (b, pi ), ⇒ αi = (Api , pi ) Note: xi above actually can be replaced by any x0 + v, v ∈ Ki (A, r0)(Why?) CS 594-0004, 04-23-2008 Conjugate Gradient Method (continued) Note: Conjugate Gradient Method One matrix vector product/iteration (at line 9) 1: Compute r0 = b − Ax0 for some initial guess x0 Two inner-products/iteration (lines 3 and 10) 2: for i = 0 to ... do In exact arithmetic r = b − Ax 3: ρ = r T r i+1 i+1 i i i (Apply A to both sides of 11 and subtract from b to 4: if i = 0 then get line 12) 5: p0 = r0 Update for x is as pointed out before, i.e. with 6: else i+1 ρi 7: pi = ri + pi−1 ρi−1 (ri , ri ) (ri , pi ) 8: end if αi = = 9: qi = Api (Api , pi ) (Api , pi ) ρi 10: αi = T pi qi since (ri , pi−1) = 0 (exercise) 11: xi+1 = xi + αi pi 12: ri+1 = ri − αi qi Other relations to be proved (exercise) 13: check convergence; continue if necessary pi s’ span the Krylov space end for 14: pi s’ are (·, ·)A orthogonal, etc. CS 594-0004, 04-23-2008 Conjugate Gradient Method (continued) To sum it up: In exact arithmetic we get the exact solution in at most n steps, i.e. x = x0 + α0p0 + ··· + αi pi + αi+1pi+1 + ··· + αn−1pn−1 At every iteration one more term αj pj is added to the current approximation xi = x0 + α0p0 + ··· + αi−1pi−1 xi+1 = x0 + α0p0 + ··· + αi−1pi−1 +αi pi ≡ xi + αi pi Note: we do not have to solve linear system at every iteration because of the A-orthogonal basis that we manage to maintain and expend at every iteration It can be proved that the error ei = x − xi satisfies !i pk(A) − 1 ||ei ||A ≤ 2 ||e0||A pk(A) + 1 CS 594-0004, 04-23-2008 Building orthogonal basis for a Krylov subspace Building orthogonal basis for a Krylov subspace We have seen the importance in Defining projections not just for linear solvers Abstract linear solvers and eigen-solver formulations A specific example in CG where the basis for the Krylov subspaces is A-orthogonal (A is SPD) We have seen how to build it CGS, MGS, Cholesky or Householder based, etc. These techniques can be used in a method specifically designed for Krylov subspaces (general non-Hermitian matrix), namely in the Arnoldi’s Method CS 594-0004, 04-23-2008 Arnoldi’s Method Note: This orthogonalization is based on CGS (line 4) Arnoldi’s method: wj = Avj − (Avj , v1)v1 − ... − (Avj , vj )vj Build an orthogonal basis for Km(A, r0) A can be general, non-Hermitian ⇒ up to iteration j vectors 1: v1 = r0 v1, ..., v 2: for j = 1 to m do j 3: hij = (Avj , vi ) for i = 1, .., j are orthogonal 4: wj = Avj − h1j v1 − ... − hjj vj The space of this orthogonal basis grows by taking the 5: hj+1,j = ||wj ||2 next vector to be Avj 6: if hj+1,j = 0 Stop wj If we do not exit at step 6 we will have 7: vj+1 = hj+1,j 8: end for Km(A, r0) = span{v1, v2,..., vm} (exercise) CS 594-0004, 04-23-2008 Arnoldi’s Method (continued) Arnoldi’s method in matrix notation Denote Vm ≡ [v1,..., vm], Hm+1 = {hij }m+1×m and by Hm the matrix Hm+1 without the last row. Note that Hm is upper Hessenberg (0s below the lower second sub-diagonal) and T AVm = VmHm + wmem T Vm AVm = Hm (exercise) CS 594-0004, 04-23-2008 Arnoldi’s Method (continued) Variations: Explained using CGS Can be implemented with MGS, Householder, etc. How to use it in linear solvers? Example with the Full Orthogonalization Method (FOM) CS 594-0004, 04-23-2008 FOM Look for solution in the form xm = x0 + ym(1)v1 + ··· + ym(m)vm ≡ x0 + Vmym FOM Petrov-Galerkin conditions will be 1: β = ||r0||2 2: Compute v1,..., vm with Arnoldi T T −1 Vm Axm = Vm b 3: ym = βHm e1 T T 4: xm = x0 + Vmym ⇒ Vm A(x0 + Vmym) = Vm b T T ⇒ Vm AVmym = Vm r0 T ⇒ Hmym = Vm r0 = βe1 which is given by steps 3 and 4 of the algorithm CS 594-0004, 04-23-2008 Restarted FOM What happens when m increases? computation grows as at least O(m2)n memory is O(mn) A remedy is to restart the algorithm, leading to restarted FOM FOM(m) 1: β = ||r0||2 2: Compute v1,..., vm with Arnoldi −1 3: ym = βHm e1 4: xm = x0 + Vmym. Stop if residual is small enough. 5: Set x0 := xm and go to 1 CS 594-0004, 04-23-2008 GMRES Generalized Minimum Residual Method (GMRES) Similar to FOM Again look for solution xm = x0 + Vmym where Vm is from the Arnoldi process (i.e. Km(A, r0)) The test conditions Wm from the abstract formulation (slide 27, Lecture 7) T T Wm AVmym = Wm r0 are Wm = AVm. The difference results in step 3 from FOM, namely −1 ym = βHm e1 being replaced by ym = argminy ||βe1 − Hm+1y||2 CS 594-0004, 04-23-2008 GMRES Similarly to FOM, GMRES can be defined with Various orthogonalizations in the Arnoldi process Restart Note: Solving the least squares (LS) problem argminy ||βe1 − Hm+1y||2 can be done with QR factorization as discussed in Lecture 7, Slide 25 CS 594-0004, 04-23-2008 Lanczos Algorithm Can we improve on Arnoldi if A is symmetric? Yes! Hm becomes symmetric so it will be just 3 diagonal the simplification of Arnoldi in this case leads to the Lanczos Algorithm Lanczos can be used in deriving CG The Lanczos Algorithm Matrix H here is 3-diagonal with diagonal r0 m 1: v1 = , β1 = 0, v0 = 0 ||r0||2 2: for j = 1 to m do hii = αi 3: wj = Avj − βj vj−1 4: αj = (wj , vj ) and off diagonal 5: wj = wj − αj vj 6: βj+1 = ||wj ||2. If βj+1 = 0 then Stop hi,i+1 = βi+1 wj 7: vj+1 = βj+1 8: end for In exact arithmetic vi s’ are orthogonal but in reality orthogonalization gets lost rapidly CS 594-0004, 04-23-2008 Choice of basis for the Krylov subspace We saw how different basis for the Krylov spaces is characteristic for various methods, e.g.
Load more

Recommended publications
  • Krylov Subspaces
    Lab 1 Krylov Subspaces Lab Objective: Discuss simple Krylov Subspace Methods for finding eigenvalues and show some interesting applications. One of the biggest difficulties in computational linear algebra is the amount of memory needed to store a large matrix and the amount of time needed to read its entries. Methods using Krylov subspaces avoid this difficulty by studying how a matrix acts on vectors, making it unnecessary in many cases to create the matrix itself. More specifically, we can construct a Krylov subspace just by knowing how a linear transformation acts on vectors, and with these subspaces we can closely approximate eigenvalues of the transformation and solutions to associated linear systems. The Arnoldi iteration is an algorithm for finding an orthonormal basis of a Krylov subspace. Its outputs can also be used to approximate the eigenvalues of the original matrix. The Arnoldi Iteration The order-N Krylov subspace of A generated by x is 2 n−1 Kn(A; x) = spanfx;Ax;A x;:::;A xg: If the vectors fx;Ax;A2x;:::;An−1xg are linearly independent, then they form a basis for Kn(A; x). However, this basis is usually far from orthogonal, and hence computations using this basis will likely be ill-conditioned. One way to find an orthonormal basis for Kn(A; x) would be to use the modi- fied Gram-Schmidt algorithm from Lab TODO on the set fx;Ax;A2x;:::;An−1xg. More efficiently, the Arnold iteration integrates the creation of fx;Ax;A2x;:::;An−1xg with the modified Gram-Schmidt algorithm, returning an orthonormal basis for Kn(A; x).
    [Show full text]
  • Accelerating the LOBPCG Method on Gpus Using a Blocked Sparse Matrix Vector Product
    Accelerating the LOBPCG method on GPUs using a blocked Sparse Matrix Vector Product Hartwig Anzt and Stanimire Tomov and Jack Dongarra Innovative Computing Lab University of Tennessee Knoxville, USA Email: [email protected], [email protected], [email protected] Abstract— the computing power of today’s supercomputers, often accel- erated by coprocessors like graphics processing units (GPUs), This paper presents a heterogeneous CPU-GPU algorithm design and optimized implementation for an entire sparse iter- becomes challenging. ative eigensolver – the Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) – starting from low-level GPU While there exist numerous efforts to adapt iterative lin- data structures and kernels to the higher-level algorithmic choices ear solvers like Krylov subspace methods to coprocessor and overall heterogeneous design. Most notably, the eigensolver technology, sparse eigensolvers have so far remained out- leverages the high-performance of a new GPU kernel developed side the main focus. A possible explanation is that many for the simultaneous multiplication of a sparse matrix and a of those combine sparse and dense linear algebra routines, set of vectors (SpMM). This is a building block that serves which makes porting them to accelerators more difficult. Aside as a backbone for not only block-Krylov, but also for other from the power method, algorithms based on the Krylov methods relying on blocking for acceleration in general. The subspace idea are among the most commonly used general heterogeneous LOBPCG developed here reveals the potential of eigensolvers [1]. When targeting symmetric positive definite this type of eigensolver by highly optimizing all of its components, eigenvalue problems, the recently developed Locally Optimal and can be viewed as a benchmark for other SpMM-dependent applications.
    [Show full text]
  • Limited Memory Block Krylov Subspace Optimization for Computing Dominant Singular Value Decompositions
    Limited Memory Block Krylov Subspace Optimization for Computing Dominant Singular Value Decompositions Xin Liu∗ Zaiwen Weny Yin Zhangz March 22, 2012 Abstract In many data-intensive applications, the use of principal component analysis (PCA) and other related techniques is ubiquitous for dimension reduction, data mining or other transformational purposes. Such transformations often require efficiently, reliably and accurately computing dominant singular value de- compositions (SVDs) of large unstructured matrices. In this paper, we propose and study a subspace optimization technique to significantly accelerate the classic simultaneous iteration method. We analyze the convergence of the proposed algorithm, and numerically compare it with several state-of-the-art SVD solvers under the MATLAB environment. Extensive computational results show that on a wide range of large unstructured matrices, the proposed algorithm can often provide improved efficiency or robustness over existing algorithms. Keywords. subspace optimization, dominant singular value decomposition, Krylov subspace, eigen- value decomposition 1 Introduction Singular value decomposition (SVD) is a fundamental and enormously useful tool in matrix computations, such as determining the pseudo-inverse, the range or null space, or the rank of a matrix, solving regular or total least squares data fitting problems, or computing low-rank approximations to a matrix, just to mention a few. The need for computing SVDs also frequently arises from diverse fields in statistics, signal processing, data mining or compression, and from various dimension-reduction models of large-scale dynamic systems. Usually, instead of acquiring all the singular values and vectors of a matrix, it suffices to compute a set of dominant (i.e., the largest) singular values and their corresponding singular vectors in order to obtain the most valuable and relevant information about the underlying dataset or system.
    [Show full text]
  • Power Method and Krylov Subspaces
    Power method and Krylov subspaces Introduction When calculating an eigenvalue of a matrix A using the power method, one starts with an initial random vector b and then computes iteratively the sequence Ab, A2b,...,An−1b normalising and storing the result in b on each iteration. The sequence converges to the eigenvector of the largest eigenvalue of A. The set of vectors 2 n−1 Kn = b, Ab, A b,...,A b , (1) where n < rank(A), is called the order-n Krylov matrix, and the subspace spanned by these vectors is called the order-n Krylov subspace. The vectors are not orthogonal but can be made so e.g. by Gram-Schmidt orthogonalisation. For the same reason that An−1b approximates the dominant eigenvector one can expect that the other orthogonalised vectors approximate the eigenvectors of the n largest eigenvalues. Krylov subspaces are the basis of several successful iterative methods in numerical linear algebra, in particular: Arnoldi and Lanczos methods for finding one (or a few) eigenvalues of a matrix; and GMRES (Generalised Minimum RESidual) method for solving systems of linear equations. These methods are particularly suitable for large sparse matrices as they avoid matrix-matrix opera- tions but rather multiply vectors by matrices and work with the resulting vectors and matrices in Krylov subspaces of modest sizes. Arnoldi iteration Arnoldi iteration is an algorithm where the order-n orthogonalised Krylov matrix Qn of a matrix A is built using stabilised Gram-Schmidt process: • start with a set Q = {q1} of one random normalised vector q1 • repeat for k =2 to n : – make a new vector qk = Aqk−1 † – orthogonalise qk to all vectors qi ∈ Q storing qi qk → hi,k−1 – normalise qk storing qk → hk,k−1 – add qk to the set Q By construction the matrix Hn made of the elements hjk is an upper Hessenberg matrix, h1,1 h1,2 h1,3 ··· h1,n h2,1 h2,2 h2,3 ··· h2,n 0 h3,2 h3,3 ··· h3,n Hn = , (2) .
    [Show full text]
  • Section 5.3: Other Krylov Subspace Methods
    Section 5.3: Other Krylov Subspace Methods Jim Lambers March 15, 2021 • So far, we have only seen Krylov subspace methods for symmetric positive definite matrices. • We now generalize these methods to arbitrary square invertible matrices. Minimum Residual Methods • In the derivation of the Conjugate Gradient Method, we made use of the Krylov subspace 2 k−1 K(r1; A; k) = spanfr1;Ar1;A r1;:::;A r1g to obtain a solution of Ax = b, where A was a symmetric positive definite matrix and (0) (0) r1 = b − Ax was the residual associated with the initial guess x . • Specifically, the Conjugate Gradient Method generated a sequence of approximate solutions x(k), k = 1; 2;:::; where each iterate had the form k (k) (0) (0) X x = x + Qkyk = x + qjyjk; k = 1; 2;:::; (1) j=1 and the columns q1; q2;:::; qk of Qk formed an orthonormal basis of the Krylov subspace K(b; A; k). • In this section, we examine whether a similar approach can be used for solving Ax = b even when the matrix A is not symmetric positive definite. (k) • That is, we seek a solution x of the form of equation (??), where again the columns of Qk form an orthonormal basis of the Krylov subspace K(b; A; k). • To generate this basis, we cannot simply generate a sequence of orthogonal polynomials using a three-term recurrence relation, as before, because the bilinear form T hf; gi = r1 f(A)g(A)r1 does not satisfy the requirements for a valid inner product when A is not symmetric positive definite.
    [Show full text]
  • A Brief Introduction to Krylov Space Methods for Solving Linear Systems
    A Brief Introduction to Krylov Space Methods for Solving Linear Systems Martin H. Gutknecht1 ETH Zurich, Seminar for Applied Mathematics [email protected] With respect to the “influence on the development and practice of science and engineering in the 20th century”, Krylov space methods are considered as one of the ten most important classes of numerical methods [1]. Large sparse linear systems of equations or large sparse matrix eigenvalue problems appear in most applications of scientific computing. Sparsity means that most elements of the matrix involved are zero. In particular, discretization of PDEs with the finite element method (FEM) or with the finite difference method (FDM) leads to such problems. In case the original problem is nonlinear, linearization by Newton’s method or a Newton-type method leads again to a linear problem. We will treat here systems of equations only, but many of the numerical methods for large eigenvalue problems are based on similar ideas as the related solvers for equations. Sparse linear systems of equations can be solved by either so-called sparse direct solvers, which are clever variations of Gauss elimination, or by iterative methods. In the last thirty years, sparse direct solvers have been tuned to perfection: on the one hand by finding strategies for permuting equations and unknowns to guarantee a stable LU decomposition and small fill-in in the triangular factors, and on the other hand by organizing the computation so that optimal use is made of the hardware, which nowadays often consists of parallel computers whose architecture favors block operations with data that are locally stored or cached.
    [Show full text]
  • Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods1
    Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods1 Richard Barrett2, Michael Berry3, Tony F. Chan4, James Demmel5, June M. Donato6, Jack Dongarra3,2, Victor Eijkhout7, Roldan Pozo8, Charles Romine9, and Henk Van der Vorst10 This document is the electronic version of the 2nd edition of the Templates book, which is available for purchase from the Society for Industrial and Applied Mathematics (http://www.siam.org/books). 1This work was supported in part by DARPA and ARO under contract number DAAL03-91-C-0047, the National Science Foundation Science and Technology Center Cooperative Agreement No. CCR-8809615, the Applied Mathematical Sciences subprogram of the Office of Energy Research, U.S. Department of Energy, under Contract DE-AC05-84OR21400, and the Stichting Nationale Computer Faciliteit (NCF) by Grant CRG 92.03. 2Computer Science and Mathematics Division, Oak Ridge National Laboratory, Oak Ridge, TN 37830- 6173. 3Department of Computer Science, University of Tennessee, Knoxville, TN 37996. 4Applied Mathematics Department, University of California, Los Angeles, CA 90024-1555. 5Computer Science Division and Mathematics Department, University of California, Berkeley, CA 94720. 6Science Applications International Corporation, Oak Ridge, TN 37831 7Texas Advanced Computing Center, The University of Texas at Austin, Austin, TX 78758 8National Institute of Standards and Technology, Gaithersburg, MD 9Office of Science and Technology Policy, Executive Office of the President 10Department of Mathematics, Utrecht University, Utrecht, the Netherlands. ii How to Use This Book We have divided this book into five main chapters. Chapter 1 gives the motivation for this book and the use of templates. Chapter 2 describes stationary and nonstationary iterative methods.
    [Show full text]
  • Iterative Methods and Sparse Linear Algebra
    Lecture 16: Iterative Methods and Sparse Linear Algebra David Bindel 25 Oct 2011 Logistics I Send me a project title and group (today, please!) I Project 2 due next Monday, Oct 31 <aside topic="proj2"> Bins of particles 2h // x bin and interaction range (y similar) int ix = (int) ( x /(2*h) ); int ixlo = (int) ( (x-h)/(2*h) ); int ixhi = (int) ( (x+h)/(2*h) ); Spatial binning and hashing I Simplest version I One linked list per bin I Can include the link in a particle struct I Fine for this project! I More sophisticated version I Hash table keyed by bin index I Scales even if occupied volume computational domain Partitioning strategies Can make each processor responsible for I A region of space I A set of particles I A set of interactions Different tradeoffs between load balance and communication. To use symmetry, or not to use symmetry? I Simplest version is prone to race conditions! I Can not use symmetry (and do twice the work) I Or update bins in two groups (even/odd columns?) I Or save contributions separately, sum later Logistical odds and ends I Parallel performance starts with serial performance I Use flags — let the compiler help you! I Can refactor memory layouts for better locality I You will need more particles to see good speedups I Overheads: open/close parallel sections, barriers. I Try -s 1e-2 (or maybe even smaller) I Careful notes and a version control system really help I I like Git’s lightweight branches here! </aside> Reminder: World of Linear Algebra I Dense methods I Direct representation of matrices with simple
    [Show full text]
  • Krylov Subspace Iteration
    the Top T HEME ARTICLE KRYLOV SUBSPACE ITERATION This survey article reviews the history and current importance of Krylov subspace iteration algorithms. ince the early 1800s, researchers have lead to approximate relationships between these considered iteration methods an attrac- parameters in neighboring gridpoints. Together tive means for approximating the solu- with the prescribed behavior at the boundary tions of large linear systems. They make gridpoints and with given sources, this leads Sthese solutions possible now that we can do re- eventually to very large linear systems of equa- alistic computer simulations. The classical itera- tions, Ax = b. The vector x is the unknown para- tion methods typically converge very slowly (and meter values in the gridpoints, b is the given in- often not at all). Around 1950, researchers real- put, and the matrix A describes the relationships ized that these methods lead to solution se- between parameters in the gridpoints. Because quences that span a subspace—the Krylov sub- these relationships are often restricted to nearby space. It was then evident how to identify much gridpoints, most matrix elements are zero. better approximate solutions, without much ad- The model becomes more accurate when we ditional computational effort. refine the grid—that is, when the distance be- When simulating a continuous event, such as tween gridpoints decreases. In a 3D simulation, the flow of a fluid through a pipe or of air around this easily leads to large systems of equations. an aircraft, researchers usually impose a grid over Even a few hundred gridpoints in each coordi- the area of interest and restrict the simulation to nate direction leads to systems with millions of the computation of relevant parameters.
    [Show full text]
  • Modeling Circulating Cavity Fields Using the Discrete Linear Canonical
    Letter Journal of the Optical Society of America A 1 Modeling circulating cavity fields using the discrete linear canonical transform A. A. CIOBANU1,2,*,D.D.B ROWN1,2,P.J.V EITCH1,2, AND D. J. OTTAWAY1,2 1Department of Physics, School of Physical Sciences and The Institute of Photonics and Advanced Sensing (IPAS), The University of Adelaide, SA, 5005, Australia 2Australian Research Council Centre of Excellence for Gravitational Wave Discovery (OzGrav) *Corresponding author: [email protected] Compiled June 8, 2021 Fabry-Perot cavities are central to many optical mea- model is determined by the resolution of the sampled grid. This surement systems. In high precision experiments, such allows non gaussian features to be accurately modelled to the as aLIGO and AdV, coupled cavities are often required resolution limit of the sampled grid. leading to complex optical dynamics, particularly when In the cartesian basis the diffraction integral becomes a linear optical imperfections are considered. We show, for the operator that couples all points in the input plane to all points in the output plane. This causes the issue that in two dimensions first time, that discrete LCTs can be used to compute cir- the size of the linear operator grows as the square of the number culating optical fields for cavities in which the optics of grid points, quickly becoming too large for practical simula- have arbitrary apertures, reflectance and transmittance tions due to memory limitations. A common optimization is to perform the diffraction integrals in the Fourier domain where profiles, and shape. We compare the predictions of LCT the convolution in the diffraction integral becomes multiplica- models with those of alternative methods.
    [Show full text]
  • A Restarted Krylov Subspace Method for the Evaluation of Matrix Functions∗
    SIAM J. NUMER. ANAL. c 2006 Society for Industrial and Applied Mathematics Vol. 44, No. 6, pp. 2481–2504 ! A RESTARTED KRYLOV SUBSPACE METHOD FOR THE EVALUATION OF MATRIX FUNCTIONS∗ MICHAEL EIERMANN† AND OLIVER G. ERNST† Abstract. We show how the Arnoldi algorithm for approximating a function of a matrix times a vector can be restarted in a manner analogous to restarted Krylov subspace methods for solving linear systems of equations. The resulting restarted algorithm reduces to other known algorithms for the reciprocal and the exponential functions. We further show that the restarted algorithm inherits the superlinear convergence property of its unrestarted counterpart for entire functions and present the results of numerical experiments. Key words. matrix function, Krylov subspace approximation, Krylov projection method, restarted Krylov subspace method, linear system of equations, initial value problem AMS subject classifications. 65F10, 65F99, 65M20 DOI. 10.1137/050633846 1. Introduction. The evaluation of n n n (1.1) f(A)b, where A C × , b C , ∈ ∈ and f : C D C is a function for which f(A) is defined, is a common compu- tational task.⊃ Besides→ the solution of linear systems of equations, which involves the reciprocal function f(λ) = 1/λ, by far the most important application is the time tλ evolution of a system under a linear operator, in which case f(λ) = ft(λ) = e and time acts as a parameter t. Other applications involving differential equations require the evaluation of (1.1) for the square root and trigonometric functions (see [8, 1]). Further applications include identification problems for semigroups involving the log- arithm (see, e.g., [29]) and lattice quantum chromodynamics simulations requiring the evaluation of the matrix sign function (see [34] and the references therein).
    [Show full text]
  • Krylov Type Methods for Large Scale Eigenvalue Computations
    University of Zagreb Department of Mathematics Doctor of Philosophy Dissertation Krylov Type Methods for Large Scale Eigenvalue Computations by Zvonimir Bujanovi´c Supervisor: prof. Zlatko Drmaˇc Zagreb, 2011. Contents Contents i Introduction iii 1 The Eigenvalue Problem 1 1.1 Notation . 1 1.2 Eigenvalues, eigenvectors and eigenspaces . 3 1.3 Applications of the eigenvalue problem . 8 1.3.1 Electromagnetic fields in cavities . 8 1.3.2 The PageRank algorithm . 12 1.4 An overview of the perturbation theory . 15 1.4.1 Perturbations of eigenvalues . 16 1.4.2 Perturbations of invariant spaces . 19 1.5 Approximations from a subspace . 22 1.5.1 Evaluating quality of a subspace . 22 1.5.2 Choosing approximations for eigenpairs . 26 2 Arnoldi-type Algorithms 35 2.1 The power method . 36 2.2 Simultaneous iteration . 41 2.2.1 The QR algorithm . 44 2.3 Krylov subspaces . 49 2.3.1 Ritz approximations from Krylov subspaces . 52 2.4 The basic Arnoldi algorithm . 57 2.4.1 The Hermitian case: the Lanczos algorithm . 59 2.5 Restarting the Arnoldi algorithm . 61 2.5.1 The implicit restart . 63 2.5.2 The Krylov-Schur algorithm . 69 2.5.3 Convergence of the restarted Arnoldi method . 71 3 Using Krylov{Schur algorithm with arbitrary shifts 75 i Contents 3.1 A more general restarting scheme . 76 3.2 Restarting and the pole placement problem . 79 3.3 Matrix balancing and the computation of the Ritz values . 85 4 Ritz values of normal matrices 91 4.1 Minimax theorem and the Cauchy interlacing property .
    [Show full text]