Short course on structured matrices D.A. Bini, Universit`adi Pisa [email protected] Journ´eesNationales de Calcul Formel (JNCF) 2014 CIRM, Luminy November 3{7, 2014 JNCF | CIRM, Luminy, November, 2014 1 D.A. Bini (Pisa) Structured matrices / 118 Outline 1 Preliminaries 2 Toeplitz matrices Applications Asymptotic spectral properties Computational issues Some matrix algebras displacement operators, fast and superfast algorithms preconditioning Wiener-Hopf factorization and matrix equations A recent application 3 Rank structured matrices Basic properties Companion matrices Application: Linearizng matrix polynomials Preliminaries Structured matrices are encountered almost everywhere The structure of a matrix reflects the peculiarity of the mathematical model that the matrix describes Exploiting matrix structures is a mandatory step for designing highly efficient ad hoc algorithms for solving computational problems Structure analysis often reveals rich and interesting theoretical properties Linear models lead naturally to matrices Some structures are evident, some other structures are more hidden Nonlinear model are usually linearized or approximated by means of linear models Preliminaries Some examples: Band matrices: locality properties, functions with compact support. Spline interpolation, finite differences Toeplitz matrices: shift invariance properties. Polynomial computations, queueing models, image restoration Displacement structures, Toeplitz-like matrices: Vandermonde, Cauchy, Hankel, Bezout, Pick Semi-separable and quasi-separable matrices: inverse of band matrices, polynomial and matrix polynomial computations, integral equations Sparse matrices: Web, Page Rank, social networks, complex networks Preliminaries In this short course we will limit ourselves to describe some computational aspect of Toeplitz matrices Rank-structured matrices and show some applications The spirit is to give the flavour of the available results with pointers to the literature Notations: F is a number field, for our purpose F 2 fR; Cg N = f0; 1; 2; 3;:::g, Z = f:::; −2; −1; 0; 1; 2;:::g T is the unit circle in the complex plane i imaginary unit such that i 2 = −1 m×n F set of m × n matrices with entries in F Toeplitz matrices [Otto Toeplitz 1881-1940] Let F be a field (F 2 fR; Cg) Z Given a bi-infinite sequence fai gi2Z 2 F and an integer n, the n × n matrix Tn = (ti;j )i;j=1;n such that ti;j = aj−i is called Toeplitz matrix 2 3 a0 a1 a2 a3 a4 6a−1 a0 a1 a2 a37 6 7 T5 = 6a−2 a−1 a0 a1 a27 6 7 4a−3 a−2 a−1 a0 a15 a−4 a−3 a−2 a−1 a0 Tn is a leading principal submatrix of the (semi) infinite Toeplitz matrix T1 = (ti;j )i;j2N, ti;j = aj−i 2 a0 a1 a2 :::3 . 6 ..7 6a−1 a0 a1 7 T1 = 6 . 7 6a a .. ..7 4 −2 −1 5 . .. .. .. Toeplitz matrices Theorem (Otto Toeplitz) The matrix T1 defines a bounded linear 2 P+1 operator in ` (N), x ! y = T1x, yi = j=0 aj−i xj if and only if ai are 1 the Fourier coefficients of a function a(z) 2 L (T) +1 X 1 Z 2π a(z) = a zn; a = a(eiθ)e−inθdθ n n 2π n=−∞ 0 In this case kT k = ess supz2Tja(z)j; where kT k := sup kTxk kxk=1 The function a(z) is called symbol associated with T1 Example Pk i If a(z) = i=−k ai z is a Laurent polynomial, then T1 is a banded Toeplitz matrix which defines a bounded linear operator Block Toeplitz matrices Let F be a field (F 2 fR; Cg) m×m Given a bi-infinite sequence fAi gi2Z, Ai 2 F and an integer n, the mn × mn matrix Tn = (ti;j )i;j=1;n such that ti;j = Aj−i is called block Toeplitz matrix 2 3 A0 A1 A2 A3 A4 6A−1 A0 A1 A2 A37 6 7 T5 = 6A−2 A−1 A0 A1 A27 6 7 4A−3 A−2 A−1 A0 A15 A−4 A−3 A−2 A−1 A0 Tn is a leading principal submatrix of the (semi) infinite block Toeplitz matrix T1 = (ti;j )i;j2N, ti;j = Aj−i 2 A0 A1 A2 :::3 . 6 ..7 6A−1 A0 A1 7 T1 = 6 . 7 6A A .. ..7 4 −2 −1 5 . .. .. .. Block Toeplitz matrices with Toeplitz blocks The infinite block Toeplitz matrix T1 defines a bounded linear operator in 2 (k) ` (N) iff the blocks Ak = (ai;j ) are the Fourier coefficients of a m×m matrix-valued function A(z): T ! C , P+1 k 1 A(z) = k=−∞ x Ak = (ai;j (z))i;j=1;m such that ai;j (x) 2 L (T) If the blocks Ai are Toeplitz themselves we have a block Toeplitz matrix with Toeplitz blocks A function a(z; w): T × T ! C having the Fourier series P+1 i j a(z; w) = i;j=−∞ ai;j z w defines an infinite block Toeplitz matrix T1 = (Aj−i ) with infinite Toeplitz blocks Ak = (ak;j−i ). T1 defines a bounded operator iff a(z; w) 2 L1 For any pair of integers n; m we may construct an n × n Toeplitz matrix Tm;n = (Aj−i )i;j=1;n with m × m Toeplitz blocks Aj−i = (ak;j−i )i;j=1;m Multilevel Toeplitz matrices d A function a : T ! C having the Fourier expansion +1 X a(z ; z ;:::; z ) = a zi1 zi2 ··· zid 1 2 d i1;i2;:::;id i1 i2 id i1;:::;id =−∞ defines a d-multilevel Toeplitz matrix: that is a block Toeplitz matrix with blocks that are themselves (d − 1)-multilevel Toeplitz matrices Generalization: Toeplitz-like matrices Let Li and Ui be lower triangular and upper triangular n × n Toeplitz matrices, respectively, where i = 1;:::; k and k is independent of n k X A = Li Ui i=1 is called a Toeplitz-like matrix If k = 2, L1 = U2 = I then A is a Toeplitz matrix. If A is an invertible Toeplitz matrix then there exist Li ; Ui , i = 1; 2 such that −1 A = L1U1 + L2U2 that is, A−1 is Toeplitz-like Applications: polynomial arithmetic Polynomial multiplication Pn i Pm i a(x) = i=0 ai x , b(x) = i=0 bi x , Pm+n i c(x) := a(x)b(x), c(x) = i=0 ci x c0 = a0b0 c1 = a0b1 + a1b0 ::: 2 3 2 3 c0 a0 6 c 7 6a a 7 6 1 7 6 1 0 7 6 . 7 6 . 7 2 3 6 . 7 6 . .. .. 7 b0 6 7 6 7 b 6 . 7 6 .. .. 7 6 1 7 6 . 7 = 6an . a07 6 . 7 6 7 6 7 6 . 7 6 . 7 6 .. .. 7 4 5 6 . 7 6 . a17 6 7 6 7 bm 6 . 7 6 .. 7 4 . 5 4 . 5 cm+n an Applications: polynomial arithmetic Polynomial division Pn i Pm i a(x) = i=0 ai x , b(x) = i=0 bi x , bm 6= 0 a(x) = b(x)q(x) + r(x), deg r(x) < m q(x) quotient, r(x) remainder of the division of a(x) by b(x) 2 3 2b 3 a0 0 2 3 6b b 7 r0 6a1 7 6 1 0 7 6 . 7 6 . .. .. 7 2 q 3 6 . 7 6 . 7 6 . 7 0 6 . 7 6 7 6 7 q 6 7 6a 7 6 .. .. 7 6 1 7 6rm−17 6 m7 = 6bm . b0 7 6 . 7 + 6 7 6 . 7 6 7 6 . 7 6 0 7 6 . 7 6 .. .. 7 4 5 6 7 6 7 6 . b1 7 6 . 7 6 . 7 6 7 qn−m 4 . 5 6 . 7 6 .. 7 4 5 4 . 5 0 an bm The last n − m + 1 equations form a triangular Toeplitz system Applications: polynomial arithmetic Polynomial division 2 3 2 3 2 3 bm bm−1 ::: b2m−n q0 am 6 .. 7 q a 6 bm . 7 6 1 7 6 m+17 6 7 6 . 7 = 6 . 7 6 .. 7 6 . 7 6 . 7 4 . bm−1 5 4 5 4 5 bm qn−m an Its solution provides the coefficients of the quotient. The remainder can be computed as a difference. 2 3 2 3 2 3 2 3 r0 a0 b0 q0 6 . 7 6 . 7 6 . .. 7 6 . 7 4 . 5 = 4 . 5 − 4 . 5 4 . 5 rm−1 am−1 bm−1 ::: b0 qn−m (in the picture n − m = m − 1) Applications: polynomial arithmetic Polynomial gcd If g(x) = gcd(a(x); b(x)), deg(g(x)) = k, deg(a(x)) = n, deg(b(x)) = m. Then there exist polynomials r(x), s(x) of degree at most m − k − 1, n − k − 1, respectively, such that (B´ezoutidentity) g(x) = a(x)r(x) + b(x)s(x) In matrix form one has the (m + n − k) × (m + n − 2k) system 2 3 2 3 a b r0 2 3 0 0 g0 6 a a b b 7 6 r1 7 . 6 1 0 1 0 7 6 7 6 . 7 6 . 7 6 . 7 6 . 7 6 . .. .. .. .. 7 6 . 7 6 7 6 7 6 7 6gk 7 6 .. .. .. .. 7 6rm−k−17 6 0 7 6 an . a0 bm . b0 7 6 7 = 6 7 6 7 6 s0 7 6 . 7 6 .. .. .. .. 7 6 7 6 . 7 6 . a1 . b1 7 6 s1 7 6 7 6 . 7 6 7 6 . 7 6 .. .. 7 6 . 7 6 . 7 4 . 5 4 . 5 4 5 0 an bm sn−k−1 Sylvester matrix Applications: polynomial arithmetic Polynomial gcd The last m + n − 2k equations provide a linear system of the kind 2 3 gk r 6 0 7 S = 6 7 6 .