MATHEMATICS: CONCEPTS, AND FOUNDATIONS – Vol. I - Matrices, Vectors, Determinants, and Linear Algebra - Tadao ODA MATRICES, VECTORS, DETERMINANTS, AND LINEAR ALGEBRA Tadao ODA Tohoku University, Japan Keywords: matrix, determinant, linear equation, Cramer’s rule, eigenvalue, Jordan canonical form, symmetric matrix, vector space, linear map Contents 1. Matrices, Vectors and their Basic Operations 1.1. Matrices 1.2. Vectors 1.3. Addition and Scalar Multiplication of Matrices 1.4. Multiplication of Matrices 2. Determinants 2.1. Square Matrices 2.2. Determinants 2.3. Cofactors and the Inverse Matrix 3. Systems of Linear Equations 3.1. Linear Equations 3.2. Cramer’s Rule 3.3. Eigenvalues of a Complex Square Matrix 3.4. Jordan Canonical Form 4. Symmetric Matrices and Quadratic Forms 4.1. Real Symmetric Matrices and Orthogonal Matrices 4.2. Hermitian Symmetric Matrices and Unitary Matrices 5. Vector Spaces and Linear Algebra 5.1. Vector spaces 5.2. Subspaces 5.3. Direct Sum of Vector Spaces 5.4. Linear Maps 5.5. Change of Bases 5.6. Properties of Linear Maps 5.7. A SystemUNESCO of Linear Equations Revisited – EOLSS 5.8. Quotient Vector Spaces 5.9. Dual Spaces 5.10. Tensor ProductSAMPLE of Vector Spaces CHAPTERS 5.11. Symmetric Product of a Vector Space 5.12. Exterior Product of a Vector Space Glossary Bibliography Biographical Sketch Summary A down-to-earth introduction of matrices and their basic operations will be followed by ©Encyclopedia of Life Support Systems (EOLSS) MATHEMATICS: CONCEPTS, AND FOUNDATIONS – Vol. I - Matrices, Vectors, Determinants, and Linear Algebra - Tadao ODA basic results on determinants, systems of linear equations, eigenvalues, real symmetric matrices and complex Hermitian symmetric matrices. Abstract vector spaces and linear maps will then be introduced. The power and merit of seemingly useless abstraction will make earlier results on matrices more transparent and easily understandable. Matrices and linear algebra play important roles in applications. Unfortunately, however, space limitation prevents description of algorithmic and computational aspects of linear algebra indispensable to applications. The readers are referred to the references listed at the end. 1. Matrices, Vectors and their Basic Operations 1.1. Matrices A matrix is a rectangular array ⎛⎞ ⎜⎟aa1112 "" a 1jn a 1 ⎜⎟ ⎜⎟ ⎜⎟aa21 22 "" a22jn a ⎜⎟ ⎜⎟##"#"# ⎜⎟ ⎜⎟ ⎜⎟aa"" a a ⎜⎟iijin1 i2 ⎜⎟ ⎜⎟##"#"# ⎜⎟ ⎜⎟ ⎜⎟aa"" a a ⎝⎠mmjmn1 m2 of entries a…a11,,mn , which are numbers or symbols. Very often, such a matrix will be denoted by a single letter such as A , thus ⎛⎞ ⎜⎟aa1112 "" a 1jn a 1 ⎜⎟ ⎜⎟ ⎜⎟aa21 22 "" a22jn a ⎜⎟ ⎜⎟##"#"# ⎜⎟ A :=⎜⎟ . ⎜⎟aa"" a a ⎜⎟iijin1 i2 ⎜⎟ ⎜⎟##"#"# ⎜⎟ ⎜⎟ ⎜⎟aa"" a a ⎝⎠mmjmn1UNESCOm2 – EOLSS The notation A = ()aij is used also, for short. In this notation, the first index i is called the row index, whileSAMPLE the second index j is called CHAPTERS the column index. Each of the horizontal arrays is called a row, thus ()()()()a11, a 12 ,, …a 1j ,, …a 1 n , a 21 , a 22 ,, …a 2 j ,, …a 2 n ,, … a i 1 , a i 2 ,, …a ij ,, …a in ,, … a m 1 , a m 2 ,, …a mj ,, …a mn are called the first row, second row,…, i -th row,…, m -th row, respectively. On the other hand, each of the vertical arrays is called a column, thus ©Encyclopedia of Life Support Systems (EOLSS) MATHEMATICS: CONCEPTS, AND FOUNDATIONS – Vol. I - Matrices, Vectors, Determinants, and Linear Algebra - Tadao ODA ⎛⎞⎛⎞⎛⎞ ⎛⎞ ⎜⎟⎜⎟aa11 12⎜⎟a1 j ⎜⎟ a 1n ⎜⎟⎜⎟⎜⎟ ⎜⎟ ⎜⎟⎜⎟⎜⎟ ⎜⎟ ⎜⎟⎜⎟aa21 22⎜⎟a2 j ⎜⎟ a 2n ⎜⎟⎜⎟⎜⎟ ⎜⎟ ⎜⎟⎜⎟##⎜⎟# ⎜⎟ # ⎜⎟⎜⎟⎜⎟ ⎜⎟ ⎜⎟⎜⎟,,,,,……⎜⎟ ⎜⎟ ⎜⎟⎜⎟aa⎜⎟a ⎜⎟ a ⎜⎟⎜⎟ii12⎜⎟ij ⎜⎟ in ⎜⎟⎜⎟⎜⎟ ⎜⎟ ⎜⎟⎜⎟##⎜⎟# ⎜⎟ # ⎜⎟⎜⎟⎜⎟ ⎜⎟ ⎜⎟⎜⎟⎜⎟ ⎜⎟ ⎜⎟⎜⎟⎜⎟a ⎜⎟ ⎝⎠⎝⎠aamm12⎝⎠mj ⎝⎠ a mn are called the first column, second column,…, j -th column, …, n -th column, respectively. Such an A is called a matrix with m rows and n columns, an ()mn, - matrix, or an mn× matrix. An ()mn, -matrix with all the entries 0 is called the zero matrix and written simply as 0 , thus ⎛⎞00" ⎜⎟ 0 :=⎜⎟#%# . ⎜⎟ ⎝⎠00" 1.2. Vectors A matrix with only one row, or only one column is called a vector, thus ()a12,,,,, a …ajn …a is a row vector, while ⎛⎞ ⎜⎟b1 ⎜⎟ ⎜⎟ ⎜⎟b2 ⎜⎟ ⎜⎟# ⎜⎟ ⎜⎟ ⎜⎟b ⎜⎟i ⎜⎟ ⎜⎟# ⎜⎟ ⎜⎟ ⎜⎟b UNESCO – EOLSS ⎝⎠m is a column vectorSAMPLE. CHAPTERS The rows and columns of an ()mn, -matrix A above are thus called, the first row vector, second row vector,…, i -th row vector,…, m -th row vector, and the first column vector, second column vector,…, j -th column vector, …, n -th column vector. A (1, 1) -matrix, i.e., a number or a symbol, is called a scalar. 1.3. Addition and Scalar Multiplication of Matrices ©Encyclopedia of Life Support Systems (EOLSS) MATHEMATICS: CONCEPTS, AND FOUNDATIONS – Vol. I - Matrices, Vectors, Determinants, and Linear Algebra - Tadao ODA The addition of two (mn, ) -matrices A = (aij ) and B = (bij ) are defined by ⎛ ⎞ ⎜ ab11++ 11 ab12 12 "" ab 1jj + 1 ab 1 nn + 1 ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ab21++ 21 ab 22 22 "" a22jj + b a 22 nn + b⎟ ⎜ ⎟ ⎜ ##"#"#⎟ ⎜ ⎟ AB+:=+()abij ij =⎜ ⎟ ⎜ ab++ ab"" ab + ab +⎟ ⎜ i11 iii22 ij ij in in ⎟ ⎜ ⎟ ⎜ ##"#"#⎟ ⎜ ⎟ ⎜ ⎟ ⎜ abab++"" ab + ab +⎟ ⎝ m11 mmm22 mj mj mn mn ⎠ when the addition of the entries makes sense. The multiplication of a scalar c with an ()mn, -matrix A = (aij ) is defined by ⎛⎞ ⎜⎟ca11 ca12 "" ca 1jn ca 1 ⎜⎟ ⎜⎟ ⎜⎟ca21 ca 22 "" ca22jn ca ⎜⎟ ⎜⎟##"#"# ⎜⎟ ccaA :=()ij = ⎜⎟ ⎜⎟ca ca"" ca ca ⎜⎟iijin1 i2 ⎜⎟ ⎜⎟##"#"# ⎜⎟ ⎜⎟ ⎜⎟ca ca"" ca ca ⎝⎠mmjmn1 m2 when the multiplication of a scalar with the entries makes sense. 1.4. Multiplication of Matrices What makes matrices most interesting and powerful is the multiplication, which does wonders as explained below. Suppose that the entries appearing in our matrices are numbers which admit multiplication. Then the multiplication AB of two matrices A and B is defined when the number of columns of A is the same as the number of rows of B . Let A = (aij ) be an (lm, ) -matrix and B = (bjk ) an (mn, ) -matrix. Then their product is the (ln, )UNESCO-matrix defined by – EOLSS SAMPLEm CHAPTERS AB :=()ccabik , with ik :=∑ ij jk , j=1 or more concretely, ©Encyclopedia of Life Support Systems (EOLSS) MATHEMATICS: CONCEPTS, AND FOUNDATIONS – Vol. I - Matrices, Vectors, Determinants, and Linear Algebra - Tadao ODA ⎛ ⎞ ⎜ ab11 11++""""" a 1m b m 1 ab 11 1 k ++ a 1 m b mk ab 11 1 n ++ a 1 m b mn ⎟ ⎜ ⎟ ⎜ ⎟ ⎜ #"#"#⎟ ⎜ ⎟ AB ⎜ ab""""" a b ab a b ab a b ⎟ =++⎜ i111 imm 1 ik 11 ++ immk in 11 ++. immn⎟ ⎜ ⎟ ⎜ #"#"#⎟ ⎜ ⎟ ⎜ ⎟ ⎜ ⎟ ⎝ abl111++""""" a lmm b 1 ab lk 11 ++ a lmmk b ab ln 11 ++ a lmmn b ⎠ Of particular interest is the product Av of an ()mn, -matrix A = (aij ) with a column vector v of size n , which is the column vector of size m defined by ⎛⎞⎛⎞⎛ ⎞ ⎜⎟aaa11"" 1jn 1 ⎜⎟⎜vavav11111++" nn ⎟ ⎜⎟⎜⎟⎜ ⎟ ⎜⎟⎜⎟⎜ ⎟ ⎜⎟#"#"#⎜⎟⎜## ⎟ ⎜⎟⎜⎟⎜ ⎟ ⎜⎟ aaa""⎜⎟⎜vavav" ⎟ ⎜⎟iijin1 ⎜⎟⎜ji=++,11 inn ⎟ ⎜⎟⎜⎟⎜ ⎟ ⎜⎟#"#"#⎜⎟⎜## ⎟ ⎜⎟⎜⎟⎜ ⎟ ⎜⎟⎜⎟⎜ ⎟ ⎜⎟aaa""⎜⎟⎜ ⎟ ⎝⎠mmjmn1 ⎝⎠⎝vavavnm11++" mnn ⎠ as well as the product uA of a row vector uu…u= ()1,,m of size m with A , which is the row vector of size n defined by ⎛⎞ ⎜⎟aaa11"" 1jn 1 ⎜⎟ ⎜⎟ ⎜⎟#"#"# ⎜⎟ ⎜⎟aaa"" ()u…u…u1,,,,im⎜⎟iijin1 ⎜⎟ ⎜⎟#"#"# ⎜⎟ ⎜⎟ ⎜⎟aaa"" ⎝⎠mmjmn1 =++,,++,,++.()ua111""" umm a 1 …ua 11 j u mmj a …ua 11 n u mmn a T The transpose A of an (mn, ) -matrix A = (aij ) is the (nm, ) -matrix defined by T A :=()aa′′ji , with ji :=aij , UNESCO – EOLSS or more concretely, SAMPLE CHAPTERS T ⎛⎞⎛ ⎞ ⎜⎟aa1112 "" a 1jn a 1 ⎜ aa1121 ⋅⋅ aim 1 ⋅⋅ a 1 ⎟ ⎜⎟⎜ ⎟ ⎜⎟⎜ ⎟ ⎜⎟aa21 22 "" a22jn a ⎜ aa12 22 ⋅⋅ aim22 ⋅⋅ a⎟ ⎜⎟⎜ ⎟ ⎜⎟##"#"# ⎜ ## # #⎟ ⎜⎟⎜ ⋅⋅ ⋅⋅ ⎟ ⎜⎟:= ⎜ ⎟ . ⎜⎟aa"" a a ⎜ aa⋅⋅ a ⋅⋅ a⎟ ⎜⎟iijin1 i2 ⎜ 1 jijmj2 j ⎟ ⎜⎟⎜ ⎟ ⎜⎟##"#"# ⎜ ##⋅⋅ # ⋅⋅ #⎟ ⎜⎟⎜ ⎟ ⎜⎟⎜⎟ ⎜⎟aa"" a a ⎜⎟ ⎝⎠mmjmn1 m2 ⎝ aa1ninmn2n ⋅⋅ a ⋅⋅ a⎠ ©Encyclopedia of Life Support Systems (EOLSS) MATHEMATICS: CONCEPTS, AND FOUNDATIONS – Vol. I - Matrices, Vectors, Determinants, and Linear Algebra - Tadao ODA For an ()lm, -matrix A and an ()mn, -matrix B , it is easy to see that ()ABTTT=, B A when the multiplication of the numbers concerned is commutative. When A and B are (nn, ) -matrices, both products AB and BA make sense, but they need not be the same in general. 2. Determinants 2.1. Square Matrices Square matrices, namely matrices with the same number of rows and columns, are most interesting. Special among them is the identity matrix of size n , denoted by I or In and defined by ⎛⎞10" ⎜⎟ II=:=#%# =()δ , nij⎜⎟ ⎜⎟ ⎝⎠01" where δij is known as Kronecker’s delta defined by ⎧1 ij= δij :=⎨ . ⎩0 ij≠ For an arbitrary (mn, ) -matrix A , the following clearly holds: IAmn== Aand AI A. The matrix cI with the same entry c along the diagonal and 0 elsewhere is called a scalar matrix. UNESCO – EOLSS More generally, a square matrix D = (dij ) of size n is called a diagonal matrix if dij = 0 for ij≠ SAMPLE, that is, CHAPTERS ⎛⎞ ⎜⎟d11 00" ⎜⎟ ⎜⎟ ⎜⎟00d22 " D =.⎜⎟ ⎜⎟#%# ⎜⎟ ⎜⎟ ⎜⎟ ⎝⎠00" dnn 2.2. Determinants ©Encyclopedia of Life Support Systems (EOLSS) MATHEMATICS: CONCEPTS, AND FOUNDATIONS – Vol. I - Matrices, Vectors, Determinants, and Linear Algebra - Tadao ODA Let A = (aij ) be a square matrix of size n (also said to be of order n ), that is, an (nn, )- matrix or an nn× matrix. When the entries aij are numbers (rational numbers, real numbers, complex numbers, or more generally elements of a commutative ring to be introduced in Rings and Modules), for which addition, subtraction and commutative multiplication are possible, associated to A is a number called the determinant of A and denoted by ||A or by det(A ) . When n =1 or n = 2 , the determinant is defined to be aa11 12 aa11:= 11 , := aaaa 11 22 − 12 21 . aa21 22 For n = 3, the formula is a bit more complicated. aaa11 12 13 aaa21 22 23 :=++−−−. aaaaaaaaaaaaaaaaaa11 22 33 12 23 31 13 21 32 13 22 31 11 23 32 12 21 33 aaa31 32 33 The determinant of A = ()aij for general n is defined as follows: aa11" 1n #%#:=∑sgn(σ )aa1(1)2(2)σσ " aii σ () " a nn σ () , σ aannn1 " where σ runs through the permutations of the indices {1, 2,,,……n }, and sgn(σ ) is the signature of σ to be defined elsewhere in Groups and Applications, since it is not so practical to compute the determinant using this formula.