Linear Algebra (VII)
Yijia Chen
1. Review
Basis and Dimension. We fix a vector space V. Lemma 1.1. Let A, B ⊆ V be two finite sets of vectors in V, possibly empty. If A is linearly indepen- dent and can be represented by B. Then |A| 6 |B|. Theorem 1.2. Let S ⊆ V and A, B ⊆ S be both maximally linearly independent in S. Then |A| = |B|.
Definition 1.3. Let e1,..., en ∈ V. Assume that
– e1,..., en are linearly independent,
– and every v ∈ V can be represented by e1,..., en.
Equivalently, {e1,..., en} is maximally linearly independent in V. Then {e1,..., en} is a basis of V. Note that n = 0 is allowed, and in that case, it is easy to see that V = {0}. By Theorem 1.2:
0 0 Lemma 1.4. If {e1,..., en} and {e1,..., em} be both bases of V with pairwise distinct ei’s and with 0 pairwise distinct ei, then n = m.
Definition 1.5. Let {e1,..., en} be a basis of V with pairwise distinct ei’s. Then the dimension of V, denoted by dim(V), is n. Equivalently, if rank(V) is defined, then dim(V) := rank(V).
1 Theorem 1.6. Assume dim(V) = n and let u1,..., un ∈ V.
(1) If u1,..., un are linearly independent, then {u1,..., un} is a basis.
(2) If every v ∈ V can be represented by u1,..., un, then {u1,..., un} is a basis.
Steinitz exchange lemma.
Theorem 1.7. Assume that dim(V) = n and v1,..., vm ∈ V with 1 6 m 6 n are linearly indepen- dent. Furthermore, let {e1,..., en} be a basis of V. Then for some 1 6 i1 < i2 < ··· < in−m 6 n
v1,..., vm, ei1 ,..., ein−m is a basis of V.
1 We do not assume beforehand that u1,..., un are pairwise distinct, although under the conditions in the theorem, they have to be, i.e., ui 6= uj for every 1 6 i < j 6 n.
1 Proof: We prove by induction on m and start with m = 1. Since {e1,..., en} is a basis, v1 can be represented by e1,..., en. Thus, there exist a1,..., an ∈ R such that
v1 = aiei. iX∈[n] As v1 6= 0 (otherwise, v1 is linearly dependent), there is an i ∈ [n] with ai 6= 0. It follows that
1 aj aj ei = v1 + − · ei + − · ei. ai ai ai i Now assume that m > 1 and v1,..., vm ∈ V are linearly independent. Of course v1,..., vm−1 are linearly independent too. By induction hypothesis on m − 1, there exist 1 6 i1 < i2 < ··· < in−m+1 6 n such that v1,..., vm−1, ei1 ,..., ein−m+1 is a basis of V. In particular, vmcan be represented by this basis, i.e., there exist a1,..., am−1, c1,..., cn−m+1 ∈ R such that vm = aivi + cjeij . i∈X[m−1] j∈[nX−m+1] Observe that j∈[n−m+1] cjeij 6= 0, otherwise, v1,..., vm would be linearly dependent. Thus, for some j ∈ [n − m + 1] we have c 6= 0, and thereby P j ai 1 c` eij = − · vi + · vm + − · ei` cj cj cj i∈X[m−1] `∈[n−Xm+1]\{j} Then it is easy to see that v1,..., vm, ei1 ,..., eij−1 , eij+1 ,..., ein−m+1 is a basis of V. 2 Remark 1.8. (i) The above proof is in fact essentially the same proof for Lemma 1.1. (ii) Again, we can drop the requirement 1 6 m, in particular, the case m = 0 holds trivially. 2. Back to the Textbook Matrix and matrix operations. Recall an m × n matrix has the form a11 a12 ··· a1n a21 a21 ··· a2n A = = a , . . .. . ij m×n . . . . am1 a21 ··· amn T where each aij ∈ R. The transpose matrix of A, denoted by A , is the n × m matrix a11 a21 ··· am1 a12 a22 ··· am2 . . . .. . . . . . a1n a2n ··· amn 2 Definition 2.1 (Matrix Addition). Let A = aij m×n and B = bij m×n be two m × n-matrices. Then a11 + b11 ··· a1n + b1n A + B := a + b = . .. . . ij ij m×n . . . am1 + bm1 ··· amn + bmn Definition 2.2. The zero m × n-matrix is 0 ··· 0 . .. . 0m×n = . . . . 0 ··· 0 When m, n are clear from the context, we write 0 instead of 0m×n. Lemma 2.3. (i) A + B = B + A. (ii) (A + B) + C = A + (B + C). (iii) (A + B)T = AT + BT . Definition 2.4 (Scalar Multiplication). Let A = aij m×n be an m × n-matrix and k ∈ R. Then k · a11 ··· k · a1n k · A := k · a = . .. . . ij m×n . . . k · am1 ··· k · amn Lemma 2.5. Let A and B be two m × n-matrices and k, ` ∈ N. (i) k · (` · A) = (k · `) · A. (ii) (k + `) · A = k · A + ` · A. (iii) k · (A + B) = k · A + k · B. (iv) (k · A)T = k · AT . Definition 2.6. For every A = aij m×n we define −A := −1 · A = − aij m×n. Lemma 2.7. A + (−A) = 0. 2.1. Matrix multiplication. Definition 2.8. Let m, n, r > 1, A be an m × r-matrix, and B an r × n-matrix. Then C := AB = cij m×n is an m × n-matrix where each cij := ai`b`j. `X∈[r] 3 Although matrix multiplication seems arbitrary at first sight, we have seen that it could be understood in the context of substituting the variables in one system of linear equations by another system of linear equations. Three matrices, A, B, and C, correspond to the coefficients of the three systems. Remark 2.9. Compared to most multiplications we have encountered before, matrix multiplica- tion is not commutative. – AB and BA can have different size, or even one of them is not defined. For instance, A = 1 1 0 4 and B = 1. Then AB is a 1 × 1-matrix, while BA is 3 × 3. 0 0 0 – Even if they have the same size, AB and BA can be different matrices. Let A := and 0 1 0 1 0 0 0 1 B := . Then AB = and BA = . 0 0 0 0 0 0 This is hardly a surprise by viewing AB as substituting variables xi’s with yi’s and BA the other way around for two systems of linear equations. Definition 2.10. For n > 1 we define 1 0 ··· 0 0 1 ··· 0 I = . m . . .. . . . . . 0 0 ··· 1 2 In is called the (n × n) identity matrix. Lemma 2.11. (i) (AB)C = A(BC). (ii) C(A + B) = CA + CB = and (A + B)C = AC + BC. (iii) k · (AB) = (k · A)B = A(k · B). (iv) Assume that A is an m × n-matrix. Then ImA = AIn = A. 3. Block Matrix For mostly computational reasons, sometimes we need to partition a matrix into several submatri- ces, or more precisely, block matrices. The following is an example. 1 2 −1 0 A A A = 2 5 0 −2 = 11 12 A A 3 1 −1 3 21 22 where 1 2 −1 0 A = , A = , 11 2 5 0 12 −2 A21 = 3 1 −1 , A22 = 3 . 2 Note that this is not the identity in the matrix space Mn×n(R), which is the scalar 1 ∈ R. 4 In general, for some p, q ∈ N with 1 6 p 6 m and 1 6 q 6 n we break m rows of A into p blocks, and similarly break n columns of A into q blocks: n1 columns nq columns m1 rows A11 A12 ··· A1q A A ··· A 21 22 2q . . .. . . . . . mp rows A A ··· A r1 p2 pq . Here, each Aij is an mi × nj-matrix for i ∈ [p] and j ∈ [q]. Remark 3.1. Note for fixed p and q, we might still have different Aij p×q, which actually de- pends on the choices of mi’s and nj’s. Example 3.2. Let 1 1 0 0 0 −1 1 0 0 0 A1 0 0 A = 0 0 1 0 0 = 0 A2 0 , 0 0 1 1 0 0 0 A3 0 0 0 0 1 where 1 1 1 0 A = , A = and A = 1 . 1 −1 1 2 1 1 3 The matrix A1 0 0 0 A2 0 0 0 A3 is a block diagonal matrix. 3.1. Operations on block matrices. Transpose. Assume A11 A12 ··· A1q A21 A22 ··· A2q A = . . . .. . . . Ap1 Ap2 ··· Apq Then T T T A11 A21 ··· Ap1 T T A12 A22 ··· Ap2 AT = . . . .. . . . T T T A1q A2q ··· Apq Addition. Let A := Aij p×q and B := Bij p×q, where for every i ∈ [p] and j ∈ [q] the matrices of Aij and Bij have the same number of rows and columns. Then it is easy to verify that A + B = Aij + Bij p×q. 5 Scalar multiplication. Let k ∈ R. Then k · A = k · Aij p×q. Matrix multiplication. Assume A11 ··· A1t B11 ··· B1q . .. . . .. . A = . . . and B = . . . . Ap1 ··· Apt Bt1 ··· Btq Moreover, for every i ∈ [p], j ∈ [t], and ` ∈ [q] the number of columns of Aij is equal to the number of rows of Bj`. In particular, Aij · Bj` is defined. Then C11 ··· C1q . .. . A · B = . . . Cp1 ··· Cpq with Ci` = Aij · Bj` jX∈[t] for any i ∈ [p] and ` ∈ [q]. Exercises Exercise 3.3. Recall that Mm×n(R) is the vector space containing all the m × n matrices with the vector addition in Definition 2.1 and with the scalar multiplication in Definition 2.4. Is Mm×n(R) finite-dimensional, and if so, what is dim Mm×n(R) ? Exercise 3.4. Consider the vector space R4. Construct a basis containing the following two vec- tors. (1, 1, 0, 1), (10, 7, 2, 3). n Exercise 3.5. Let A be an m × n matrix. Assume that for every column vector x ∈ R , viewed as an n × 1 matrix, Ax = 0. Prove that A = 0m×n. Exercise 3.6. Assume x1 y1 x2 y2 x = and y = . . . . xn yn T k Let A = yx . Please calculate A , i.e., A · A ··· A, for every k > 1. k times | {z } 6