LINEAR ALGEBRA T.K.SUBRAHMONIAN MOOTHATHU Contents 1. Vector spaces and subspaces 2 2. Linear independence 5 3. Span, basis, and dimension 7 4. Linear operators 11 5. Linear functionals and the dual space 15 6. Linear operators = matrices 18 7. Multiplying a matrix left and right 22 8. Solving linear equations 28 9. Eigenvalues, eigenvectors, and the triangular form 32 10. Inner product spaces 36 11. Orthogonal projections and least square solutions 39 12. Unitary operators: they preserve distances and angles 43 13. Orthogonal diagonalization of normal and self-adjoint operators 46 14. Singular value decomposition 50 15. Determinant of a square matrix: properties 51 16. Determinant: existence and expressions 54 17. Minimal and characteristic polynomials 58 18. Primary decomposition theorem 61 19. The Jordan canonical form (JCF) 65 20. Trace of a square matrix 71 21. Quadratic forms 73 Developing effective techniques to solve systems of linear equations is one important goal of Linear Algebra. Our study of Linear Algebra will proceed through two parallel roads: (i) the theory of linear operators, and (ii) the theory of matrices. Why do we say parallel roads? Well, here is an illustrative example. Consider the following system of linear equations: 1 2 T.K.SUBRAHMONIAN MOOTHATHU 2x1 + 3x2 = 7 4x1 − x2 = 5. 2 We are interested in two questions: is there a solution (x1; x2) 2 R to the above? and if there is a solution, is the solution unique? This problem can be formulated in two other ways: 2 2 First reformulation: Let T : R ! R be T (x1; x2) = (2x1 + 3x2; 4x1 − x2). Does (7; 5) belong to the range of T ? and if so, is the pre-image T −1(7; 5) a singleton? 2 3 2 3 2 3 7 Second reformulation: Let A = 4 5 and Y = 4 5. Does there exist a real matrix 4 −1 5 2 3 x1 X = 4 5 such that AX = Y ? and if so, is X unique? x2 The first reformulation is in terms of a linear operator T and the second reformulation is in terms of a matrix A. Often, a statement in terms of linear operators has an equivalent version in terms of matrices, and vice versa. That is why we say the two theories are parallel roads. A student may keep the following challenge in mind throughout the course. When you encounter a result about linear operators, see whether you can formulate a corresponding result about matrices, and vice versa. If you are successful in translating the result, try to provide a proof also. 1. Vector spaces and subspaces We assume that the student has an elementary knowledge of groups and fields, and some famil- iarity with addition and multiplication of matrices (excluding the theory of determinants). Definition: Let F be a field. An abelian group (V; +) is said to be a vector space over F if there is a map ∗ : F × V ! V called scalar multiplication satisfying the following: (i) 1 ∗ v = v for every v 2 V , where 1 is the multiplicative identity of the field F . (ii) c ∗ (d ∗ v) = (cd) ∗ v for every v 2 V and c; d 2 F . (iii) (c + d) ∗ v = c ∗ v + d ∗ v for every v 2 V and c; d 2 F . (iv) c ∗ (u + v) = c ∗ u + c ∗ v for every u; v 2 V and c 2 F . Elements of V are called vectors and elements of F are called scalars. Remark: In the above, if F = R, then V is said to be a real vector space; and if F = C, then V is said to be a complex vector space. Example: (i) If F is a field and n 2 N, then F n is a vector space over F with respect to the coordinatewise operations given as (x1; : : : ; xn) + (y1; : : : ; yn) = (x1 + y1; : : : ; xn + yn) and n n c(x1; : : : ; xn) = (cx1; : : : ; cxn). In particular, R is a real vector space, and C is a complex vector LINEAR ALGEBRA 3 space. Also F 0 := f0g is a vector space over F . (ii) If F is a subfield of a field K, then K is a vector space over F under the natural operations. For instance, R is a vector space over Q, and C is a vector space over R. (iii) If F is a field, then the polynomial ring fall polynomials with coefficients in F g is a vector space over F under the operations (f + g)(x) = f(x) + g(x) and (cf)(x) = cf(x). Remark: Linear Algebra is mainly concerned with finite dimensional vector spaces. Later we will see that any finite dimensional vector space over a field F is isomorphic to F n for some n ≥ 0. If (G; +) is an abelian group and c 2 G, then we can define nc 2 G for any n 2 Z by the following rules: 0 · c := 0, 1 · c = c,(n + 1) · c := n · c + c and (−n) · c := n · (−c) for n 2 N. Exercise-1: Let V be a vector space over a field F , and let v 2 V , c 2 F . Then, (i) 0 ∗ v = 0 2 V and (−1) ∗ v = −v. More generally, (nc) ∗ v = c ∗ (nv) for every n 2 Z. (ii) If v =6 0 and c ∗ v = 0 2 V , then c = 0 2 F .[Hint: (i) 0 ∗ v = (0 + 0) ∗ v = 0 ∗ v + 0 ∗ v and use the cancellation law in the group (V; +). Next, 0 = 0 ∗ v = (1 + (−1)) ∗ v. (ii) Multiply by c−1.] Remark: From now onwards we drop the ∗ symbol and just write cv instead of c ∗ v. Also it is convenient to use the following notations: if V is a vector space over a field F , v 2 V , and W; Z ⊂ V , let F v = fcv : c 2 F g, v + W = fv + w : w 2 W g, FW = fcw : c 2 F; w 2 W g, and W + Z = fw + z : w 2 W; z 2 Zg. In particular, we have W + W = fw1 + w2 : w1; w2 2 W g. Next we talk about ways to form new vector spaces. In Group Theory, three easy methods to produce new groups are: take subgroups, form quotient groups, and form the product of some groups. Similarly, in Linear Algebra, three easy methods to produce new vector spaces are: take vector subspaces, form quotient vector spaces, and form the product of some vector spaces. Definition: Let (V; +) be a vector space over a field F . A subgroup W of (V; +) is said to be a vector subspace of V if FW ⊂ W . Generally, we will use the equivalent condition (ii) given below. Exercise-2: Let (V; +) be a vector space over a field F and W ⊂ V be a nonempty set. Then the following are equivalent: (i) W is a vector subspace of V . (ii) W + W ⊂ W and FW ⊂ W , i.e., w1 + w2 2 W and cw1 2 W for every w1; w2 2 W and c 2 F . (iii) FW + W ⊂ W , i.e., cw1 + w2 2 W for every w1; w2 2 W and c 2 F . (iv) FW + FW ⊂ W , i.e., cw1 + dw2 2 W for every w1; w2 2 W and c; d 2 F . Example: Any line passing through the origin and any plane containing the origin are proper vector subspaces of R3. The remaining vector subspaces of R3 are the trivial ones, f0g and R3. Exercise-3: Let V is a vector space over a field F . Then, 4 T.K.SUBRAHMONIAN MOOTHATHU (i) If v 2 V , then F v is a vector subspace of V . (ii) The intersection of any collection of vector subspaces of V is again a vector subspace of V . What about the union? (iii) If W1;:::;Wn ⊂ V are vector subspaces of V , then W1 + ··· + Wn is a vector subspace of V . In finite dimension, vector subspaces are related to homogeneous systems of linear equations, whereas various function spaces have infinite dimensional vector subspaces; see below. Definition: Let F be a field and consider a system of m linear equations in n variables over F (here, `over F ' means the constants aij; yi below are from F ): a11x1 + ··· + a1nxn = y1 a21x1 + ··· + a2nxn = y2 . a x + ··· + a x = y . m1 1 mn n m 2 3 2 3 6x17 6 y1 7 6 7 6 7 6 . 7 6 . 7 In matrix form, this is AX = Y , where A = [aij]m×n, X = 4 . 5, and Y = 4 . 5. Often it is xn ym n m convenient to make the identifications X ∼ x = (x1; : : : ; xn) 2 F and Y ∼ y = (y1; : : : ; ym) 2 F . Then we may write Ax = y for AX = Y . The set fx 2 F n : Ax = yg is the solution space of the system AX = Y . The system AX = Y is homogeneous if Y = 0, and non-homogeneous if Y =6 0. Example:[Finite dimensional] Let F be a field and AX = 0 be a homogeneous system of m linear equations in n variables over F . Verify that the solution space W := fx 2 F n : Ax = 0g is a vector subspace of F n. Later we will see that every vector subspace of F n is of this form. A general principle: We are yet to define dimension, but the following remark can be kept in mind. Typically (but not always) each linear equation reduces the dimension of the solution space by 1. If AX = 0 is a homogeneous system of m linear equations in n variables over a field F , and if m ≤ n, then typically (but not always) the solution space W will have dimension n − m.