Fast algorithms with prepro cessing for matrixvector multiplication problems IGohberg and VOlshevsky School of Mathematical Sciences Raymond and Beverly Sackler Faculty of Exact Sciences Tel Aviv University Ramat Aviv Israel email gohb ergmathtauacil vadimmathtauacil Journal of Complexity Abstract In this pap er the problem of complexityofmultiplication of a matrix with a vector is studied for To eplitz Hankel Vandermonde and Cauchy matrices and for matrices connected with them ie for transp ose inverse and transp ose to inverse matrices The prop osed algorithms have complexities at most O n log n ops and in a number of cases improv e the known estimates In these algorithms in a separate preprocessing phase are singled out all the actions on the preparation of a given matrix which aimed at the reduction of the complexity of the second stage of computations directly connected with the multiplication by an arbitrary vector Incidentally the eective algorithms for computing the Vandermonde determinant and the determinant of a Cauchy matrix are given Intro duction nn Let the matrix A C be given by all its n entries The problem is to compute the n pro duct Ab of a matrix A by input vector b C Using the standard rule of a matrix times vector multiplication the ab ove problem can be solved in n n ops ie oat point op erations of addition subtraction multiplication and division In the following three simple examples the sp ecial structure of a matrix enables faster computation of the pro duct by avector Examples Band matrices n Let A a b e a band matrix with the width of the band ie by denition a ij ij ij Obviously in this case the computing the pro duct Ab costs n while j i j j ops Smal l rank matrices nn Let matrix A C with rank R be given in the form of the outer sum of terms X n T h g C h g A m m m m m The representation shows that A can b e multiplied by an arbitrary vector in n ops Semiseparable matrices n Such a matrix A a is dened by the equalities ij ij P f g ij mi mi m a ij i j n n n where f f g g m are given vectors from C In this m mi m mi i i case X diagg A diagf m m m where by diagf is denoted the diagonal matrix whose entries on the main diagonal equal n the co ordinates of the vector f C The pro duct of a semiseparable matrix by an arbitrary vector can b e computed using in n ops Obviously the analogous estimate holds for the transp ose to the matrices of the form nn Let matrix A C be given The task is to compute the pro ducts Ab Ab of the n matrix A by input vectors b b C in the smallest p ossible time Such a situation arises naturally in a numb er of computational problems for example when wehave to carry out the iterations with a given matrix The problem of computing the pro duct of a matrix by a matrix can also b e solved in the ab ove framework In the latter case the columns of a second matrix are interpreted as input vectors In accordance with the accepted scheme we single out all the computations which are not dep endent up on the input vectors b b Accordingly the prop osed algorithms will b e divided into two separate phases nn I Prepro cessing for matrix A C II Application to the vector The rst phase also contains the preparation of the given matrix which enables the second phase to b e accomplished more eectively In the present pap er weembody this scheme and prop ose a numb er of fast algorithms for matrices with a certain structure namely for transp osed Vandermonde matrix for transp ose to inverse of Vandermonde matrix for Cauchy matrices and for matrices connected with them ie for transp ose inverse and transp ose to inverse matrices Background Basic algorithms In this pap er a limited number of well known algorithms is used intensively These algorithms are listed below and accompanied by the estimates of their complexities The particular implementations of these basic algorithms and complexity analysis for them can b e found in various sources see for example Aho Hop croft and Ullman BA Evaluation algorithm An algorithm of evaluation of a n degree p olynomial at n points The complexity of this algorithm will b e denoted by n It is well kno wn that n O n log n BA Interpolation algorithm An algorithm of interp olation of a n degree p olynomial from its values at n points The complexity of the interp olation algorithm is denoted by n and as it is well known n O n log n BA Fast Fourier Transform algorithm n n The discrete Fourier transform of the vector r r C is by denition the vector i i P n n n ik C where is the primitive nth ro ot from unity The discrete r k i k Fourier transform can be computed by well known metho ds collectively named Fast Fourier Transform FFT It is well known that for the complexity n of computing one FFT of order n the following estimate holds n O n log n Basic matrices In this subsection some matrices which are known to havealower than O n ops complex ityofmultiplication with a vector are considered Note that the metho ds for fast computing of the pro ducts of these basic matrices byvectors are based on the algorithms BA BA BM Vandermonde matrix By denition the Vandermonde matrix V t is the matrix of the form n t t n t t n n t t C with V t i i n t t n n Obviously the problem of computing the pro duct V tb of the matrix V tbyavector P n n i b b is equivalent to the problem of evaluation of p b at n p oints i i i i t t t Thus the pro duct V tb can b e computed using the algorithm BA n in n ops BM Inverse of Vandermonde matrix nn Consider the problem of application to a vector of the matrix A C which is n dened as the inverse of the given Vandermonde matrix V t t C It is easy to see that the pro duct Ab can be computed using the algorithm BA Thus for the inverse of Vandermonde matrix the application to avector costs n ops BM Fourier matrix and its inverse i n be primitive nth ro ot from the unity Consider the Fourier matrix Let e n i p F V with and its inverse F F where sup erscript means i n conjugate transp ose The pro ducts F b and F b can b e computed using the algorithm BA in nops BM Factor circulant n By Circ r will b e denoted circulant with the rst column r r ie matrix i i of the form r r r n r r r n Circ r r r n r r r n The matrix Circ r is referred to as a circulant It is known see Cline Plemmons and Worm that the matrix Circ r admits the following decomp osition r F D Circ rD F n i where F is the Fourier matrix r diagF D r and D diag with i n C satisfying the condition From it follows that if the co ordinates of n n i the vector C are given then the pro duct of the matrix Circ r by an i arbitrary vector can b e computed in nO n ops In this case the prepro cessing phase consists of computation of the central factor rinthe righthand side of and costs nO n ops BM Toeplitz matrices n The pro duct of To eplitz matrix A a by a vector can be computed by em ij ij b edding of the matrix A in the circulantofdouble size Corresp ondingly the amount of op erations is nO nnO n ops after n O n nO n prepro cessing BM Inverses of Toeplitz matrices In computations with the inverses of To eplitz matrices the Gohb ergSemencul formula see Gohb erg and Semencul or Gohberg and Feldman is useful This formula represents the inverse of a To eplitz matrix in the form of the sum of pro ducts of triangular To eplitz matrices Namely if for the To eplitz matrix T the equations T x e T y e n h i h i T T n with e e have solutions x x n i i n y y and x then T is invertible and i i y y y x n n y y x x n n T x x y n y x x x n n x x n y x n y x n y y n On the basis of formula the numb er of fast algorithms for the inversion of Toeplitz matrices was elab orated see Brent Gustavson and Yun de Ho og Chun and Kailath where the metho ds for solving the equations of the form are suggested Denote by n the complexity of solving one equation of the form According to Brent Gustavson and Yun de Ho og Chun and Kailath n O n log n nn Thus if matrix A C is the inverse of the given To eplitz matrix then from the ab ove arguments follows that after nn ops prepro cessing the pro duct of A by an arbitrary vector can be computed by in n O n ops Below we show how this complexity can be reduced nn n If for To eplitz matrix T C the equations have solutions x x y i i n y and x then i i T Circ x Circ Z y Circ Z y Circ x x where numbers and are arbitrary and Z is the cyclic lower shift matrix Formula was obtained in Ammar and Gader for p ositive denite To eplitz matrices and and was extended to the general case in Gohberg and Olshevsky Note in the latter pap er one can also nd other factor circulant decomp ositions for the inverses of To eplitz matri ces which are useful in the fast computations with To eplitz matrices Furthermore representing each factor circulant in the lefthand side of in the form of we nally get T D F x F D D F Z y x Z y F D D F x F D n where F is a Fourier matrix and the diagonal