Sp ecial Matrices Complex Matrices mn A C implies that a are complex numbers ij a i ij ij ij ij ij i m j n Op erations are the same except for Transp osition if a i then ij ij ij H T C A A c a i ij ji ji ji n Inner pro duct if x y C then n X H s x y x y i i i H x x is real n n X X H x x x x jx j i i i i i Band Matrices mn Denition A has lower bandwidth p if a ij for i j p and upper bandwidth q if a for j i q ij A The denotes an arbitrary nonzero entry This matrix has lower bandwidth and upp er band width Example A upp er triangular matrix p q Some Band Matrices Matrix p q Diagonal Upp er Triangular n Lower Triangular m Tridiagonal Upp er bidiagonal Lower bidiagonal Upp er Hessenberg n Lower Hessenberg m Example A tridiagonal matrix p q A Example A Lower Hessenberg matrix p q A Lower by Upp er Triangular Matrix Pro duct Example cf Golub and Van Loan Problem p L and U are n n lower and upp er triangular matrices Give a column saxpy algorithm for computing C UL l u u u c c c l l u u l l l u Express the columns of C as a linear combination of the columns of U u u u l l l c u u u u u u l u l c u l u c u u Lower by Upp er Triangular Matrix Pro duct Use the saxpy matrix multiplication formula n X c u l j k kj k l if k j kj n X c u l j k kj k j u if i k ik function C suplowU L suplow compute the matrix pro duct C UL using saxpys where L and U are n n lower and upp er triangular matrices n n sizeU C zerosn for j n for k jn Ckj saxpyLkj Ukk Ckj end end Lower by Upp er Triangular Matrix Pro duct Op eration count A saxpy for a vector of length k requires k multiplications and k additions The inner k lo op has n j saxpys on vectors of length k and requires n X k k j multiplications and additions Recall n X nn i i The multiplicationsadditions in the inner lo op are j n n X X X nn j j k k k k j k k The multiplicationsadditions in the outer lo op are n X nn j j j Recall n X n n n i i Lower by Upp er Triangular Matrix Pro duct Thus the total op eration count is n n n nn n n or n n n n n n For large n there are approximately n multiplications and additions Multiplication of two n n full matrices requires n multipli cations and additions BandMatrix Storage nn Let A have lower and upp er bandwidth p and q If p q n then store A in either a p q n or a band n p q matrix A to save storage Example Consider a matrix with p and q a a a a a a a a a a a a a a A a a a a a a a a a a a a a a a a a a a Row Storage slide all rows to the left or right to align the columns on antidiagonals a a a a a a a a a a a a a a band A a a a a a a a a a a a a a a a a a a a band npq A with band a a ij ij ip Can also store A by columns Band MatrixVector Multiplication Example Multiply a band matrix A by a vector x minniq minniq X X band y a x a x i ij j j ij ip j maxip j maxip band function y bmatvecp q A x bmatvec compute the matrixvector pro duct y Ax band where A is an n n matrix stored in banded form by rows and x is an nvector The scalars p and q are the lower and upp er bandwidths of A resp ectively start and stop indicate the column index of the rst and band last nonzero entries in row i of A jstart and jstop indicate similar column indices for A n n sizeA y n for i n jstart max ip start jstart i p jstop minn iq stop jstop i p band yi dotA istartstop xjstartjstop end Symmetric Matrics nn T Denition A is symmetric if A A A Only half of the matrix need be stored Store the lower or upp er triangular part of A as a vector sy m a Store the upp er triangular part of A by rows sy m a a a ij iniij Arrange storage so that inner lo ops access contiguous data FORTRAN stores matrices by columns C stores matrices by rows Symmetric matrix by vector multiplication n i n X X X a x a x a x b y ij j ji j ij j i j j j i Can also store A by diagonals Blo ck Matrices mn Partition the rows and columns of A into submatrices such that A A A q A A A q A A A A p p pq m n i j A R i m j n ij m m m m n n n n p q All op erations on matrices with scalar comp onents apply to those with matrix comp onents Matrix op erations on the blo cks are required k l Let B B B B s B B B s B B B B r r rs k l i j B i k j l ij k k k k l l l l r s Blo ck Matrix Addition and Multiplication Addition a partition is conformable for addition if m k n l p r q s m k n l i m j l i i j j Then C A B with C A B i p ij ij ij j q Multiplication a partition is conformable for multiplication if n k q r n k j q j j Then C C C s C C C s C AB a C C C p p ps with q X C A B i p j s b ij it tj t Sparse Matrices Denition Sparse matrices have a signicant number of zeros A matrix is considered to be sparse if the zeros are neither to be stored nor op erated up on Motivation Sparse matrices with arbitrary zero structures arise in nite element and network problems A Storage Store A as a vector a by rows or columns An integer vector ja stores the column indices of A when A is stored by rows A second vector ia stores the indices in ja of the rst element of each row of A The last element of ia ia usually signals termi n nation Sparse Matrices For example a ja ia The dimension of a and ja is the number of nonzeros in A That of ia is n ia is set to the b eginning index of a cticious row n n The storage scheme is called the compressed sparse row CSR format CSC is the compressed sparse column format Linked structures can be used instead of arrays Sparse Matrices Example Multiply a sparse matrix A by a vector x function y smatveca ja ia x smatvec compute the matrixvector pro duct y Ax where a is the nbyn matrix A stored in CSR format ja is a vector of pointers to the nonzero elements of A ia is a vector pointing to the rst nonzero element in each row of A n lengthia for i n start and stop lo cate the b eginning and end of row i in a and ja start iai stop iai yi for k startstop yi yi akxjak end end cf Y Saad Iterative Methods for Sparse Linear Systems PWS Boston Section .