DOCSLIB.ORG

Circulant and Toeplitz Matrices in Compressed Sensing

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Circulant and Toeplitz Matrices in Compressed Sensing Circulant and Toeplitz Matrices in Compressed Sensing Holger Rauhut Hausdorff Center for Mathematics & Institute for Numerical Simulation University of Bonn Conference on Time-Frequency Strobl, June 15, 2009 Institute for Numerical Simulation Rheinische Friedrich-Wilhelms-Universität Bonn Holger Rauhut Hausdorff Center for Mathematics & Institute for NumericalCirculant and Simulation Toeplitz University Matrices inof CompressedBonn Sensing Overview • Compressive Sensing (compressive sampling) • Partial Random Circulant and Toeplitz Matrices • Recovery result for `1-minimization • Numerical experiments • Proof sketch (Non-commutative Khintchine inequality) Holger Rauhut Circulant and Toeplitz Matrices 2 Recovery requires the solution of the underdetermined linear system Ax = y: Idea: Sparsity allows recovery of the correct x. Suitable matrices A allow the use of efficient algorithms. Compressive Sensing N Recover a large vector x 2 C from a small number of linear n×N measurements y = Ax with A 2 C a suitable measurement matrix. Usual assumption x is sparse, that is, kxk0 := j supp xj ≤ k where supp x = fj; xj 6= 0g. Interesting case k < n << N. Holger Rauhut Circulant and Toeplitz Matrices 3 Compressive Sensing N Recover a large vector x 2 C from a small number of linear n×N measurements y = Ax with A 2 C a suitable measurement matrix. Usual assumption x is sparse, that is, kxk0 := j supp xj ≤ k where supp x = fj; xj 6= 0g. Interesting case k < n << N. Recovery requires the solution of the underdetermined linear system Ax = y: Idea: Sparsity allows recovery of the correct x. Suitable matrices A allow the use of efficient algorithms. Holger Rauhut Circulant and Toeplitz Matrices 3 n×N For suitable matrices A 2 C , `0 recovers every k-sparse x provided n ≥ 2k. Problem: `0 is NP hard in general. Need efficient alternative algorithms! We will consider `1-minimization `0-minimization Natural approach min kxk0 subject to Ax = y Holger Rauhut Circulant and Toeplitz Matrices 4 Problem: `0 is NP hard in general. Need efficient alternative algorithms! We will consider `1-minimization `0-minimization Natural approach min kxk0 subject to Ax = y n×N For suitable matrices A 2 C , `0 recovers every k-sparse x provided n ≥ 2k. Holger Rauhut Circulant and Toeplitz Matrices 4 Need efficient alternative algorithms! We will consider `1-minimization `0-minimization Natural approach min kxk0 subject to Ax = y n×N For suitable matrices A 2 C , `0 recovers every k-sparse x provided n ≥ 2k. Problem: `0 is NP hard in general. Holger Rauhut Circulant and Toeplitz Matrices 4 `0-minimization Natural approach min kxk0 subject to Ax = y n×N For suitable matrices A 2 C , `0 recovers every k-sparse x provided n ≥ 2k. Problem: `0 is NP hard in general. Need efficient alternative algorithms! We will consider `1-minimization Holger Rauhut Circulant and Toeplitz Matrices 4 `1-minimization Convex relaxation of `0 problem: N X min kxk1 = jxj j subject to Ax = y j=1 Efficient convex optimization methods apply. Idea: The minimizers of `1 and `0 coincide under appropriate conditions on the matrix A and on the sparsity k of x. Holger Rauhut Circulant and Toeplitz Matrices 5 Theorem (Cand´es,Romberg, Tao, '06,'08; Foucart, Lai '08) p n×N 2(3− 2) IfA 2 C is a matrix satisfying δ2k < 7 ≈ 0:4531 then `1-minimization stably recovery everyk-sparsex fromy = Ax. Restricted Isometry Property (RIP) The restricted isometry constant δk of a matrix A is the smallest constant δ such that 2 2 2 (1 − δ)kxk2 ≤ kAxk2 ≤ (1 + δ)kxk2 for all k-sparse x: All column submatrices of A with at most k columns are well-conditioned. Holger Rauhut Circulant and Toeplitz Matrices 6 Restricted Isometry Property (RIP) The restricted isometry constant δk of a matrix A is the smallest constant δ such that 2 2 2 (1 − δ)kxk2 ≤ kAxk2 ≤ (1 + δ)kxk2 for all k-sparse x: All column submatrices of A with at most k columns are well-conditioned. Theorem (Cand´es,Romberg, Tao, '06,'08; Foucart, Lai '08) p n×N 2(3− 2) IfA 2 C is a matrix satisfying δ2k < 7 ≈ 0:4531 then `1-minimization stably recovery everyk-sparsex fromy = Ax. Holger Rauhut Circulant and Toeplitz Matrices 6 Random Partial Fourier matrix: (n rows of the discrete Fourier matrix selected at random) n ≥ Cδ−2k log4(N) log(−1): RIP for random matrices n×N A Bernoulli random matrix A 2 R with independent entries p1 having values ± n with equal probability; or a Gaussian random n×N matrix A 2 R having independent normal distributed entries with mean zero and variance 1=n satisfies δk ≤ δ with probability at least1 − provided n ≥ Cδ−2(k log(N=k) + log(−1)): Holger Rauhut Circulant and Toeplitz Matrices 7 RIP for random matrices n×N A Bernoulli random matrix A 2 R with independent entries p1 having values ± n with equal probability; or a Gaussian random n×N matrix A 2 R having independent normal distributed entries with mean zero and variance 1=n satisfies δk ≤ δ with probability at least1 − provided n ≥ Cδ−2(k log(N=k) + log(−1)): Random Partial Fourier matrix: (n rows of the discrete Fourier matrix selected at random) n ≥ Cδ−2k log4(N) log(−1): Holger Rauhut Circulant and Toeplitz Matrices 7 Circulant and Toeplitz matrices N Circulant matrix: For b = (b0; b1;:::; bN−1) 2 R let b N×N S = S 2 R be the matrix with entries Si;j = bj−i mod N , 0 1 b0 b1 ······ bN−1 BbN−1 b0 b1 ··· bN−2C Sb = B C : B . C @ . A b1 b2 ··· bN−1 b0 2N−1 Toeplitz matrix: For c = (c−N−1; c−N−2;:::; cN−1) 2 R let c N×N T = T 2 R be the matrix with entries Ti;j = ci−j , 0 1 c0 c1 ······ cN−1 B c−1 c0 c1 ··· cN−2C T c = B C : B . C @ . A c−N−1 c−N−2 ··· c−1 c0 Holger Rauhut Circulant and Toeplitz Matrices 8 N 2N−1 We will always choose the vectors b 2 R ; c 2 R at random. b c Performance of SΩ, TΩ in compressed sensing? Partial random circulant and Toeplitz matrices LetΩ ⊂ f1;:::; Ng arbitrary of cardinality n. N RΩ: operator that restricts a vector x 2 R to its entries in Ω. Restrict Sb and T c to the rows indexed by Ω: b b n×N Partial circulant matrix: SΩ = RΩS 2 R c c n×N Partial Toeplitz matrix: TΩ = RΩT 2 R Convolution followed by subsampling: b y = RΩS x = RΩ(x ∗ b) Holger Rauhut Circulant and Toeplitz Matrices 9 Partial random circulant and Toeplitz matrices LetΩ ⊂ f1;:::; Ng arbitrary of cardinality n. N RΩ: operator that restricts a vector x 2 R to its entries in Ω. Restrict Sb and T c to the rows indexed by Ω: b b n×N Partial circulant matrix: SΩ = RΩS 2 R c c n×N Partial Toeplitz matrix: TΩ = RΩT 2 R Convolution followed by subsampling: b y = RΩS x = RΩ(x ∗ b) N 2N−1 We will always choose the vectors b 2 R ; c 2 R at random. b c Performance of SΩ, TΩ in compressed sensing? Holger Rauhut Circulant and Toeplitz Matrices 9 Why Circulant and Toeplitz matrices? • Structure: Fast algorithms via the FFT • Smaller amount of randomness than for Gaussian / Bernoulli matrices • Natural appearance in applications such as system identification, radar, channel estimation in wireless communication, ... Holger Rauhut Circulant and Toeplitz Matrices 10 Previous results Bajwa et al.(2007/2008): Ω = f1;:::; ng, the entries of b and c are either independent Bernoulli ±1 or standard normal random variables. 2 p1 b p1 c If n ≥ Cδk log(N=k) then n SΩ and n TΩ satisfy RIP, δ2k ≤ δ In N particular, `1-minimization recovers k-sparse vectors x 2 R . Romberg (2008): Ω is chosen at random, b = F −1b^, where F is the Fourier transform and the entries of b^ are essentially chosen at random from a uniform distribution on the torus (such that Sb is a real matrix). 3 Assume n ≥ maxfC1k log(N/); C2 log (N/)g. Fix a support set S and choose the signs of the non-zero coefficients of x on S independently and uniformly at random. Then `1-minimization recovers x with probability at least1 − . Holger Rauhut Circulant and Toeplitz Matrices 11 Main Result Theorem Let Ω ⊂ f1; 2;:::; Ng be an arbitrary (deterministic) set of N cardinality n. Letx 2 R bek-sparse such that the signs of its non-zero entries are Bernoulli ±1 random variables. Choose N b 2 R to be a random vector whose entries are ±1 Bernoulli b n random variables. Lety = SΩx 2 R . There exists a constant C > 0 such that n ≥ Ck log3(N/) implies that with probability at least 1 − the `1-minimizer coincides withx. c b The same statement holds withT Ω in place of SΩ where 2N−1 c 2 R is a random vector with Bernoulli ±1 entries. Improvements with respect to previous results: Theorem removes randomness in Ω, n scales essentially linear in k c and it works also for TΩ. Holger Rauhut Circulant and Toeplitz Matrices 12 Numerical experiments Partial Circulant and Toeplitz matrix generated by Bernoulli vector, N = 500, n = 100. Empirical Recovery Rate Empirical Recovery Rate 1 1 0.9 0.9 0.8 0.8 0.7 0.7 0.6 0.6 0.5 0.5 0.4 0.4 0.3 0.3 0.2 0.2 0.1 0.1 0 0 0 5 10 15 20 25 30 35 40 0 5 10 15 20 25 30 35 40 Sparsity Sparsity (a) Circulant (b) Toeplitz Holger Rauhut Circulant and Toeplitz Matrices 13 Numerical experiments Partial Circulant and Toeplitz matrix generated by Gaussian vector, N = 500, n = 100.
Load more

Recommended publications
  • Problems in Abstract Algebra
    STUDENT MATHEMATICAL LIBRARY Volume 82 Problems in Abstract Algebra A. R. Wadsworth 10.1090/stml/082 STUDENT MATHEMATICAL LIBRARY Volume 82 Problems in Abstract Algebra A. R. Wadsworth American Mathematical Society Providence, Rhode Island Editorial Board Satyan L. Devadoss John Stillwell (Chair) Erica Flapan Serge Tabachnikov 2010 Mathematics Subject Classification. Primary 00A07, 12-01, 13-01, 15-01, 20-01. For additional information and updates on this book, visit www.ams.org/bookpages/stml-82 Library of Congress Cataloging-in-Publication Data Names: Wadsworth, Adrian R., 1947– Title: Problems in abstract algebra / A. R. Wadsworth. Description: Providence, Rhode Island: American Mathematical Society, [2017] | Series: Student mathematical library; volume 82 | Includes bibliographical references and index. Identifiers: LCCN 2016057500 | ISBN 9781470435837 (alk. paper) Subjects: LCSH: Algebra, Abstract – Textbooks. | AMS: General – General and miscellaneous specific topics – Problem books. msc | Field theory and polyno- mials – Instructional exposition (textbooks, tutorial papers, etc.). msc | Com- mutative algebra – Instructional exposition (textbooks, tutorial papers, etc.). msc | Linear and multilinear algebra; matrix theory – Instructional exposition (textbooks, tutorial papers, etc.). msc | Group theory and generalizations – Instructional exposition (textbooks, tutorial papers, etc.). msc Classification: LCC QA162 .W33 2017 | DDC 512/.02–dc23 LC record available at https://lccn.loc.gov/2016057500 Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting for them, are permitted to make fair use of the material, such as to copy select pages for use in teaching or research. Permission is granted to quote brief passages from this publication in reviews, provided the customary acknowledgment of the source is given. Republication, systematic copying, or multiple reproduction of any material in this publication is permitted only under license from the American Mathematical Society.
    [Show full text]
  • Analytical Solution of the Symmetric Circulant Tridiagonal Linear System
    Rose-Hulman Institute of Technology Rose-Hulman Scholar Mathematical Sciences Technical Reports (MSTR) Mathematics 8-24-2014 Analytical Solution of the Symmetric Circulant Tridiagonal Linear System Sean A. Broughton Rose-Hulman Institute of Technology, [email protected] Jeffery J. Leader Rose-Hulman Institute of Technology, [email protected] Follow this and additional works at: https://scholar.rose-hulman.edu/math_mstr Part of the Numerical Analysis and Computation Commons Recommended Citation Broughton, Sean A. and Leader, Jeffery J., "Analytical Solution of the Symmetric Circulant Tridiagonal Linear System" (2014). Mathematical Sciences Technical Reports (MSTR). 103. https://scholar.rose-hulman.edu/math_mstr/103 This Article is brought to you for free and open access by the Mathematics at Rose-Hulman Scholar. It has been accepted for inclusion in Mathematical Sciences Technical Reports (MSTR) by an authorized administrator of Rose-Hulman Scholar. For more information, please contact [email protected]. Analytical Solution of the Symmetric Circulant Tridiagonal Linear System S. Allen Broughton and Jeffery J. Leader Mathematical Sciences Technical Report Series MSTR 14-02 August 24, 2014 Department of Mathematics Rose-Hulman Institute of Technology http://www.rose-hulman.edu/math.aspx Fax (812)-877-8333 Phone (812)-877-8193 Analytical Solution of the Symmetric Circulant Tridiagonal Linear System S. Allen Broughton Rose-Hulman Institute of Technology Jeffery J. Leader Rose-Hulman Institute of Technology August 24, 2014 Abstract A circulant tridiagonal system is a special type of Toeplitz system that appears in a variety of problems in scientific computation. In this paper we give a formula for the inverse of a symmetric circulant tridiagonal matrix as a product of a circulant matrix and its transpose, and discuss the utility of this approach for solving the associated system.
    [Show full text]
  • Discovering Transforms: a Tutorial on Circulant Matrices, Circular Convolution, and the Discrete Fourier Transform
    DISCOVERING TRANSFORMS: A TUTORIAL ON CIRCULANT MATRICES, CIRCULAR CONVOLUTION, AND THE DISCRETE FOURIER TRANSFORM BASSAM BAMIEH∗ Key words. Discrete Fourier Transform, Circulant Matrix, Circular Convolution, Simultaneous Diagonalization of Matrices, Group Representations AMS subject classifications. 42-01,15-01, 42A85, 15A18, 15A27 Abstract. How could the Fourier and other transforms be naturally discovered if one didn't know how to postulate them? In the case of the Discrete Fourier Transform (DFT), we show how it arises naturally out of analysis of circulant matrices. In particular, the DFT can be derived as the change of basis that simultaneously diagonalizes all circulant matrices. In this way, the DFT arises naturally from a linear algebra question about a set of matrices. Rather than thinking of the DFT as a signal transform, it is more natural to think of it as a single change of basis that renders an entire set of mutually-commuting matrices into simple, diagonal forms. The DFT can then be \discovered" by solving the eigenvalue/eigenvector problem for a special element in that set. A brief outline is given of how this line of thinking can be generalized to families of linear operators, leading to the discovery of the other common Fourier-type transforms, as well as its connections with group representations theory. 1. Introduction. The Fourier transform in all its forms is ubiquitous. Its many useful properties are introduced early on in Mathematics, Science and Engineering curricula [1]. Typically, it is introduced as a transformation on functions or signals, and then its many useful properties are easily derived. Those properties are then shown to be remarkably effective in solving certain differential equations, or in an- alyzing the action of time-invariant linear dynamical systems, amongst many other uses.
    [Show full text]
  • Linear Algebra: Example Sheet 3 of 4
    Michaelmas Term 2020 Linear Algebra: Example Sheet 3 of 4 1. Find the eigenvalues and give bases for the eigenspaces of the following complex matrices: 0 1 1 0 1 0 1 1 −1 1 0 1 1 −1 1 @ 0 3 −2 A ; @ 0 3 −2 A ; @ −1 3 −1 A : 0 1 0 0 1 0 −1 1 1 The second and third matrices commute; find a basis with respect to which they are both diagonal. 2. By considering the rank or minimal polynomial of a suitable matrix, find the eigenvalues of the n × n matrix A with each diagonal entry equal to λ and all other entries 1. Hence write down the determinant of A. 3. Let A be an n × n matrix all the entries of which are real. Show that the minimum polynomial of A, over the complex numbers, has real coefficients. 4. (i) Let V be a vector space, let π1; π2; : : : ; πk be endomorphisms of V such that idV = π1 + ··· + πk and πiπj = 0 for any i 6= j. Show that V = U1 ⊕ · · · ⊕ Uk, where Uj = Im(πj). (ii) Let α be an endomorphism of V satisfying the equation α3 = α. By finding suitable endomorphisms of V depending on α, use (i) to prove that V = V0 ⊕ V1 ⊕ V−1, where Vλ is the λ-eigenspace of α. (iii) Apply (i) to prove the Generalised Eigenspace Decomposition Theorem stated in lectures. [Hint: It may help to re-read the proof of the Diagonalisability Theorem given in lectures. You may use the fact that if p1(t); : : : ; pk(t) 2 C[t] are non-zero polynomials with no common factor, Euclid's algorithm implies that there exist polynomials q1(t); : : : ; qk(t) 2 C[t] such that p1(t)q1(t) + ::: + pk(t)qk(t) = 1.] 5.
    [Show full text]
  • Factoring Matrices Into the Product of Circulant and Diagonal Matrices
    FACTORING MATRICES INTO THE PRODUCT OF CIRCULANT AND DIAGONAL MATRICES MARKO HUHTANEN∗ AND ALLAN PERAM¨ AKI¨ y n×n Abstract. A generic matrix A 2 C is shown to be the product of circulant and diagonal matrices with the number of factors being 2n−1 at most. The demonstration is constructive, relying on first factoring matrix subspaces equivalent to polynomials in a permutation matrix over diagonal matrices into linear factors. For the linear factors, the sum of two scaled permutations is factored into the product of a circulant matrix and two diagonal matrices. Extending the monomial group, both low degree and sparse polynomials in a permutation matrix over diagonal matrices, together with their permutation equivalences, constitute a fundamental sparse matrix structure. Matrix analysis gets largely done polynomially, in terms of permutations only. Key words. circulant matrix, diagonal matrix, DFT, Fourier optics, sparsity structure, matrix factoring, polynomial factoring, multiplicative Fourier compression AMS subject classifications. 15A23, 65T50, 65F50, 12D05 1. Introduction. There exists an elegant result, motivated by applications in optical image processing, stating that any matrix A 2 Cn×n is the product of circu- lant and diagonal matrices [15, 18].1 In this paper it is shown that, generically, 2n − 1 factors suffice. (For various aspects of matrix factoring, see [13].) The demonstration is constructive, relying on first factoring matrix subspaces equivalent to polynomials in a permutation matrix over diagonal matrices into linear factors. This is achieved by solving structured systems of polynomial equations. Located on the borderline between commutative and noncommutative algebra, such subspaces are shown to constitute a fundamental sparse matrix structure of polynomial type extending, e.g., band matrices.
    [Show full text]
  • On the Asymptotic Equivalence of Circulant and Toeplitz Matrices Zhihui Zhu and Michael B
    1 On the Asymptotic Equivalence of Circulant and Toeplitz Matrices Zhihui Zhu and Michael B. Wakin Abstract—Any sequence of uniformly bounded N × N Her- Hermitian Toeplitz matrices fHN g by defining a function mitian Toeplitz matrices fHN g is asymptotically equivalent to a eh(f) 2 L2([0; 1]) with Fourier series2 certain sequence of N ×N circulant matrices fCN g derivedp from Z 1 the Toeplitz matrices in the sense that kHN − CN kF = o( N) −j2πkf as N ! 1. This implies that certain collective behaviors of the h[k] = eh(f)e df; k 2 Z; 0 eigenvalues of each Toeplitz matrix are reflected in those of the 1 X corresponding circulant matrix and supports the utilization of h(f) = h[k]ej2πkf ; f 2 [0; 1]: the computationally efficient fast Fourier transform (instead of e the Karhunen-Loeve` transform) in applications like coding and k=−∞ filtering. In this paper, we study the asymptotic performance of the individual eigenvalue estimates. We show that the asymptotic Usually eh(f) is referred to as the symbol or generating equivalence of the circulant and Toeplitz matrices implies the function for the N × N Toeplitz matrices fHN g. individual asymptotic convergence of the eigenvalues for certain Suppose eh 2 L1([0; 1]). Szego’s˝ theorem [1] states that types of Toeplitz matrices. We also show that these estimates N−1 asymptotically approximate the largest and smallest eigenvalues 1 X Z 1 lim #(λl(HN )) = #(eh(f))df; (1) for more general classes of Toeplitz matrices. N!1 N l=0 0 Keywords—Szego’s˝ theorem, Toeplitz matrices, circulant matri- ces, asymptotic equivalence, Fourier analysis, eigenvalue estimates where # is any function continuous on the range of eh.
    [Show full text]
  • Linear Circulant Matrices Over a Finite Field
    Available online at www.sciencedirect.com Applied J SCIENCE @DIRECT e Mathematics Letters ELSEVIER Applied Mathematics Letters 17 (2004) 1349-1355 www.elsevier.com/locate/aml Linear Circulant Matrices Over A Finite Field C. H. COOKE Department of Mathematics Old Dominion University Norfolk, VA 23529, U.S.A. (Received August 2003; revised and accepted April 200g) Abstract--Rings of polynomials RN = Zp[x]/x N - 1 which are isomorphic to Z N are studied, where p is prime and N is an integer. If I is an ideal in RN, the code/( whose vectors constitute the isomorphic image of I is a linear cyclic code. If I is a principle ideal and K contains only the trivial cycle {0} and one nontrivial cycle of maximal least period N, then the code words of K/{0} obtained by removing the zero vector can be arranged in an order which constitutes a linear circulant matrix, C. The distribution of the elements of C is such that it forms the cyclic core of a generalized Hadamard matrix over the additive group of Zp. A necessary condition that C = K/{0} be linear circulant is that for each row vector v of C, the periodic infinite sequence a(v) produced by cycling the elements of v be period invariant under an arbitrary permutation of the elements of the first period. The necessary and sufficient condition that C be linear circulant is that the dual ideal generated by the parity check polynomial h(x) of K be maximal (a nontriviM, prime ideal of RN), with N = pk _ 1 and k = deg (h(x)).
    [Show full text]
  • Block Circulant Matrices and the Spectra of Multivariate Stationary
    Spec. Matrices 2021; 9:36–51 Research Article Open Access Special Issue: Combinatorial Matrices Marianna Bolla*, Tamás Szabados, Máté Baranyi, and Fatma Abdelkhalek Block circulant matrices and the spectra of multivariate stationary sequences https://doi.org/10.1515/spma-2020-0121 Received October 30, 2020; accepted December 16, 2020 Abstract: Given a weakly stationary, multivariate time series with absolutely summable autocovariances, asymptotic relation is proved between the eigenvalues of the block Toeplitz matrix of the rst n autocovari- ances and the union of spectra of the spectral density matrices at the n Fourier frequencies, as n ! ∞. For the proof, eigenvalues and eigenvectors of block circulant matrices are used. The proved theorem has impor- tant consequences as for the analogies between the time and frequency domain calculations. In particular, the complex principal components are used for low-rank approximation of the process; whereas, the block Cholesky decomposition of the block Toeplitz matrix gives rise to dimension reduction within the innovation subspaces. The results are illustrated on a nancial time series. Keywords: weakly stationary time series, autocovariance and spectral density matrix, nite Fourier trans- form, complex principal components, block Cholesky decomposition MSC: 42B05, 62M15, 65F30 1 Introduction 2 Let fXtg be a weakly stationary, real-valued time series (t 2 Z). Assume that E(Xt) = 0, c(0) = E(Xt ) > 0, and the sequence of autocovariances c(h), (h = 1, 2, ... ) is absolutely summable; obviously, c(h) = c(−h). n Then it is known that the Toeplitz matrix Cn = [c(i − j)]i,j=1, that is the covariance matrix of the random T T vector (X1, ..
    [Show full text]
  • LNCS 8632, Pp
    A Fast Sparse Block Circulant Matrix Vector Product Eloy Romero, Andr´es Tom´as, Antonio Soriano, and Ignacio Blanquer Instituto de Instrumentaci´on para Imagen Molecular (I3M), Centro Mixto CSIC – Universitat Polit`ecnica de Val`encia–CIEMAT, Camino de Vera s/n, 46022 Valencia, Spain {elroal,antodo,asoriano}@i3m.upv.es, [email protected] Abstract. In the context of computed tomography (CT), iterative im- age reconstruction techniques are gaining attention because high-quality images are becoming computationally feasible. They involve the solution of large systems of equations, whose cost is dominated by the sparse ma- trix vector product (SpMV). Our work considers the case of the sparse matrices being block circulant, which arises when taking advantage of the rotational symmetry in the tomographic system. Besides the straight- forward storage saving, we exploit the circulant structure to rewrite the poor-performance SpMVs into a high-performance product between sparse and dense matrices. This paper describes the implementations developed for multi-core CPUs and GPUs, and presents experimental results with typical CT matrices. The presented approach is up to ten times faster than without exploiting the circulant structure. Keywords: Circulant matrix, sparse matrix, matrix vector product, GPU, multi-core, computed tomography. 1 Introduction Iterative approaches to image reconstruction have cut down the radiation dose delivered to the patient in computed tomography (CT) explorations [5], because they are less sensitive to noisy acquisitions than filtered backprojection (FBP). Iterative methods consider the reconstruction problem as a system of linear equations y = Ax. The probability matrix A constitutes a mathematical model of the tomographic system that links the reconstructed attenuation map x with the estimation of the measurement y.
    [Show full text]
  • A New Approach to Multilinear Dynamical Systems and Control*
    A New Approach to Multilinear Dynamical Systems and Control* Randy C. Hoover1, Kyle Caudle2 and Karen Braman2 Abstract—The current paper presents a new approach to mul- algorithms have played a central role in extending many of the tilinear dynamical systems analysis and control. The approach existing machine learning algorithms [1]–[12]. However, as is based upon recent developments in tensor decompositions their popularity has gained more traction over the last decade, and a newly defined algebra of circulants. In particular, it is shown that under the right tensor multiplication operator, a they have made their way into the dynamical systems and third order tensor can be written as a product of third order controls community as well [13]–[25]. While most applica- tensors that is analogous to a traditional matrix eigenvalue de- tions of Tucker/CP in the dynamical systems and controls composition where the “eigenvectors” become eigenmatrices and community revolve around the reduction of certain classes the “eigenvalues” become eigen-tuples. This new development of nonlinear systems to multilinear counterparts [15]–[21], allows for a proper tensor eigenvalue decomposition to be defined and has natural extension to linear systems theory through a [25], others have focused on time-series modeling [24], [26], tensor-exponential. Through this framework we extend many fuzzy inference [23], or identification/modeling of inverse of traditional techniques used in linear system theory to their dynamics [22]. multilinear counterpart. In the current paper, we describe a new approach to multilinear dynamical systems analysis and control through I. INTRODUCTION Fourier theory and an algebra of circulants as outlined in [27]– Traditional approaches to the analysis and control of lin- [31].
    [Show full text]
  • Circulant-Matrices
    Circulant-Matrices September 7, 2017 In [1]: using PyPlot, Interact 1 Circulant Matrices In this lecture, I want to introduce you to a new type of matrix: circulant matrices. Like Hermitian matrices, they have orthonormal eigenvectors, but unlike Hermitian matrices we know exactly what their eigenvectors are! Moreover, their eigenvectors are closely related to the famous Fourier transform and Fourier series. Even more importantly, it turns out that circulant matrices and the eigenvectors lend themselves to incredibly efficient algorithms called FFTs, that play a central role in much of computational science and engineering. Consider a system of n identical masses m connected by springs k, sliding around a circle without friction. d2~s Similar to lecture 26, the vector ~s of displacements satifies m dt2 = −kA~s, where A is the n × n matrix: 0 2 −1 −11 B−1 2 −1 C B C B −1 2 −1 C B C B .. .. .. C A = B . C B C B −1 2 −1 C B C @ −1 2 −1A −1 −1 2 (This matrix is real-symmetric and, less obviously, positive semidefinite. So, it should have orthogonal eigenvectors and real eigenvalues λ ≥ 0.) For example, if n = 7: In [2]:A=[2-10000-1 -12-10000 0-12-1000 00-12-100 000-12-10 0000-12-1 -10000-12] https://github.com/stevengj/1806-spring17/raw/master/lectures/cyclic-springs.png Figure 1: a ring of springs 1 Out[2]:7 ×7 ArrayfInt64,2g: 2 -1 0 0 0 0 -1 -1 2 -1 0 0 0 0 0 -1 2 -1 0 0 0 0 0 -1 2 -1 0 0 0 0 0 -1 2 -1 0 0 0 0 0 -1 2 -1 -1 0 0 0 0 -1 2 This matrix has a very special pattern: every row is the same as the previous row, just shifted to the right by 1 (wrapping around \cyclically" at the edges).
    [Show full text]
  • Differential Equations and Linear Algebra
    Index A beat, 128 absolute stability, 188 bell-shaped curve, 16, 189, 458 absolute value, 83, 86 Bernoulli equation, 61 acceleration, 73, 478 Bessel function, 364, 460, 478 accuracy, 183, 185, 190, 191 better notation, 113, 124, 125 Adams method, 191, 192 big picture, 298, 301, 304, 397 addition formula, 87 Black-Scholes, 457 add exponents, 9 block matrix, 230, 236, 418 adjacency matrix, 316, 318, 425 block multiplication, 225, 226 boundaryconditions, 417, 403, 409, 429, 457 Airy’s equation, 130 boundary value problem, 403, 457, 470 albedo, 49 box, 175 amplitude, 75, 82, 111 box function, 404, 437, 443, 469, 478, 488 amplitude response, 34, 77 Brauer, 179 antisymmetric, 244, 321, 349, 406 applied mathematics, 314, 421, 487 C arrows, 155, 316 capacitance, 119 associative law, 219 carbon, 46 attractor, 169, 180 carrying capacity, 53, 55, 61 augmented matrix, 230, 257, 271, 278 Castillo-Chavez, 179 autocorrelation, 480 catalyst, 179 autonomous, 57, 71, 156, 157, 159 Cayley-Hamilton theorem, 345 average, 434, 438 cell phone, 44, 175 center, 160, 162, 173 B centered difference, 6, 189 back substitution, 212, 262 chain rule, 3, 4, 365, 368 backslash, 220 change of variables, 362 backward difference, 6, 12, 245, 413 chaos, 154, 180 backward Euler, 187, 188 characteristic equation, 90, 103, 108, 163 bad news, 326 chebfun, 402 balance equation, 48, 118, 314, 424 chemical engineering, 457 balance of forces, 118 chess matrix, 309 bank, 12, 40, 485 Cholesky factorization, 400 bar, 403, 405, 409, 455, 457 circulant matrix, 204, 448, 486, 488 basis,
    [Show full text]