Sp ecial Matrices
Complex Matrices
m n
A C implies that a are complex numbers
ij
a i
ij ij ij ij ij
i m j n
Op erations are the same except for
Transp osition if a i then
ij ij ij
H T
C A A c a i
ij ji ji ji
n
Inner pro duct if x y C then
n
X
H
s x y x y
i i
i
H
x x is real
n n
X X
H
x x x x jx j
i i i
i i
Band Matrices
m n
De nition A has lower bandwidth p if a
ij
for i j p and upper bandwidth q if a for j i q
ij
A
The denotes an arbitrary nonzero entry
This matrix has lower bandwidth and upp er band
width
Example A upp er triangular matrix p q
Some Band Matrices
Matrix p q
Diagonal
Upp er Triangular n
Lower Triangular m
Tridiagonal
Upp er bidiagonal
Lower bidiagonal
Upp er Hessenberg n
Lower Hessenberg m
Example A tridiagonal matrix p q
A
Example A Lower Hessenberg matrix p q
A
Lower by Upp er
Triangular Matrix Pro duct
Example cf Golub and Van Loan Problem
p
L and U are n n lower and upp er triangular matrices
Give a column saxpy algorithm for computing
C UL
l u u u
c c c
l l u u
l l l u
Express the columns of C as a linear combination of the
columns of U
u u u
l l l c
u u
u
u u u
l u l c u l u c
u u
Lower by Upp er
Triangular Matrix Pro duct
Use the saxpy matrix multiplication formula
n
X
c u l
j k kj
k
l if k j
kj
n
X
c u l
j k kj
k j
u if i k
ik
function C suplow U L
suplow compute the matrix pro duct C UL using saxpys
where L and U are n n lower and upp er
triangular matrices
n n size U
C zeros n
for j n
for k j n
C k j saxpy L k j U k k C k j
end
end
Lower by Upp er
Triangular Matrix Pro duct
Op eration count
A saxpy for a vector of length k requires k multiplications
and k additions
The inner k lo op has n j saxpys on vectors of length k
and requires
n
X
k
k j
multiplications and additions
Recall
n
X
n n
i
i
The multiplications additions in the inner lo op are
j
n n
X X X
n n j j k k k
k j k k
The multiplications additions in the outer lo op are
n
X
n n j j
j
Recall
n
X
n n n
i
i
Lower by Upp er
Triangular Matrix Pro duct
Thus the total op eration count is
n n n n n
n n
or
n n n n n
n
For large n there are approximately n multiplications
and additions
Multiplication of two n n full matrices requires n multipli
cations and additions
Band Matrix Storage
n n
Let A have lower and upp er bandwidth p and q
If p q n then store A in either a p q n or a
band
n p q matrix A to save storage
Example Consider a matrix with p and q
a a
a a a
a a a a
a a a a a
A
a a a a a
a a a a a
a a a a a
a a a a
Row Storage slide all rows to the left or right to align
the columns on anti diagonals
a a
a a a
a a a a
a a a a a
band
A
a a a a a
a a a a a
a a a a a
a a a a
band n p q
A with
band
a a
ij
i j i p
Can also store A by columns
Band Matrix Vector Multiplication
Example Multiply a band matrix A by a vector x
min n i q min n i q
X X
band
y a x a x
i ij j j
i j i p
j max ip j max ip
band
function y bmatvec p q A x
bmatvec compute the matrix vector pro duct y Ax
band
where A is an n n matrix stored in banded
form by rows and x is an n vector The scalars p and q
are the lower and upp er bandwidths of A resp ectively
start and stop indicate the column index of the rst and
band
last nonzero entries in row i of A
jstart and jstop indicate similar column indices for A
n n size A
y n
for i n
jstart max i p
start jstart i p
jstop min n i q
stop jstop i p
band
y i dot A i start stop x jstart jstop
end
Symmetric Matrics
n n T
De nition A is symmetric if A A
A
Only half of the matrix need be stored
Store the lower or upp er triangular part of A as a vector
sy m
a
Store the upp er triangular part of A by rows
sy m
a a a
ij
i ni i j
Arrange storage so that inner lo ops access contiguous data
FORTRAN stores matrices by columns
C stores matrices by rows
Symmetric matrix by vector multiplication
n i n
X X X
a x a x a x b y
ij j ji j ij j i
j j j i
Can also store A by diagonals
Blo ck Matrices
m n
Partition the rows and columns of A into submatrices
such that
A A A
q
A A A
q
A
A A A
p p pq
m n
i j
A R i m j n
ij
m m m m n n n n
p q
All op erations on matrices with scalar comp onents apply
to those with matrix comp onents
Matrix op erations on the blo cks are required
k l
Let B
B B B
s
B B B
s
B
B B B
r r rs
k l
i j
B i k j l
ij
k k k k l l l l
r s
Blo ck Matrix
Addition and Multiplication
Addition a partition is conformable for addition if
m k n l p r q s
m k n l i m j l
i i j j
Then C A B with C A B i p
ij ij ij
j q
Multiplication a partition is conformable for multiplication
if
n k q r n k j q
j j
Then
C C C
s
C C C
s
C AB a
C C C
p p ps
with
q
X
C A B i p j s b
ij it tj
t
Sparse Matrices
De nition
Sparse matrices have a signi cant number of zeros
A matrix is considered to be sparse if the zeros are neither
to be stored nor op erated up on
Motivation
Sparse matrices with arbitrary zero structures arise in
nite element and network problems
A
Storage
Store A as a vector a by rows or columns
An integer vector ja stores the column indices of A
when A is stored by rows
A second vector ia stores the indices in ja of the rst
element of each row of A
The last element of ia ia usually signals termi
n
nation
Sparse Matrices
For example
a ja ia
The dimension of a and ja is the number of nonzeros in A
That of ia is n
ia is set to the b eginning index of a cticious row
n
n
The storage scheme is called the compressed sparse row CSR
format
CSC is the compressed sparse column format
Linked structures can be used instead of arrays
Sparse Matrices
Example Multiply a sparse matrix A by a vector x
function y smatvec a ja ia x
smatvec compute the matrix vector pro duct y Ax
where a is the n by n matrix A stored in CSR
format ja is a vector of pointers to the nonzero elements
of A ia is a vector pointing to the rst nonzero
element in each row of A
n length ia
for i n
start and stop lo cate the b eginning
and end of row i in a and ja
start ia i
stop ia i
y i
for k start stop
y i y i a k x ja k
end
end
cf Y Saad Iterative Methods for Sparse Linear
Systems PWS Boston Section