Lesson 3: Matrix products, transpose 1/7 1 2 3 4 5 6 7 New concepts: Matrix-matrix product ¡ Matrix transpose ¡ Transpose of matrix sums, products ¡ Scalar product of two vectors ¡ Matrix-matrix product 2/7 1 2 3 4 5 6 7
m n n p De nition. Consider matrices A = ( a1 ::: an ) R , and X = x1 ::: xp R : 2 m p 2 The matrix product B = AX is a matrix B = b1 ::: bp R ¡ with column vectors given by the matrix vector products 2 ¡
bk = Axk; for k = 1; 2:::; p:
A matrix-matrix product is simply a set of matrix-vector products, and hence expresses multiple linear combinations in a concise way. The dimensions of the matrices must be compatible, the number of rows of X must equal the number of columns of A. A matrix-vector product is a special case of a matrix-matrix product when p = 1. We often write B = AX in terms of columns as
b1 ::: bp = A x1 ::: xp = Ax1 ::: Axp ¡ ¡ ¡ Matrix transpose expressed through components 3/7 1 2 3 4 5 6 7
m n De nition. Given a matrix A R , 2
a11 a12 a1n a a a A =0 21 22 2n 1 B a a a C B m1 m2 mn C @ A T n m the transpose of A, denoted as A R is 2
a11 a21 an1 a a a AT =0 12 22 n2 1: B a a a C B 1m 2m nm C @ A
Intuitively: rows become columns, columns become rows Matrix transpose expressed through vectors 4/7 1 2 3 4 5 6 7
m 1 Recall that a vector is the special case of a matrix with a single column, v R . The T 1 m 2 transpose of a vector is v R a matrix with a single row, known as a row vector. 2 m n Given a matrix A R expressed through its column vectors 2
A = ( a1 a2 ::: an );
its transpose can be expressed as
T a1 T T a2 n m A =0 1 R : 2 B T C B an C B C @ A Transpose of vector sums, matrix sums 5/7 1 2 3 4 5 6 7 For u; v Rm, (u + v)T = uT + vT . Proof: by direct computation 2
T T u1 v1 u1 + v1 (u + v)T = + = = ( u + v ::: u + v ) 00 1 0 11 0 1 1 1 m m um vm um + vm @@ A @ AA @ A T T =( u1 ::: um ) + ( v1 ::: vm ) = u + v
m n T T T For A; B R , (A + B) = A + B . Proof: by direct computation using column vectors of 2A; B
A = ( a1 ::: an ); B = ( b1 ::: bn )
T T T (a1 + b1) a1 + b1 T T T T (A + B) = ( a1 + b1 ::: an + bn ) =0 1=0 1= A + B T T T (an + bn) an + bn B C B C @ A @ A Transpose of matrix-vector, matrix-matrix products 6/7 1 2 3 4 5 6 7
m n n T Consider A R , x R . What is (Ax) ? Recall that Ax is a linear combination of columns o2f A 2
Ax = ( a1 ::: an )x = x1a1 + ::: + xnan
Take transpose of vector sum to obtain
T a1 T T T T T (Ax) = x1a1 + ::: + xnan = ( x1 ::: xn )0 1= x A (1) T an B C @ A m n n p T T T Now consider A R ; B R . Obtain (AB) = B A by applying (1) 2 2
T T T T (Ab1) b1 A b1 T T T T T (AB) = ( Ab1 ::: Abn ) =0 1=0 1=0 1A = B A T T T T (Abn) bn A bn B C B C B C @ A @ A @ A Scalar product of two real-valued vectors 7/7 1 2 3 4 5 6 7 A special case of a matrix-matrix product occurs when the two factors correspond to a row multiplying a column vector. The result is in this case a single scalar.
De nition. Given two real-valued (column) vectors u; v Rm the scalar product is the matrix-matrix multiplication 2
v1 m T v2 u v = ( u1 u2 ::: um )0 1= uivi i=1 B v C X B m C @ A
m 1 T 1 m m 1 Note the compatibility of number of components, u R u R ,v R , uTv R1 1 = R. 2 ) 2 2 2 The scalar product can be viewed as function taking two vectors as arguments and pro- ducing a single scalar as a result. The usual notation in this case is
m ; : R; u ; v = uTv = u v h i V V ! h i i i i=1 X with = Rm. V