Linear Algebra
Total Page:16
File Type:pdf, Size:1020Kb
Linear Algebra Dave Penneys August 7, 2008 2 Contents 1 Background Material 7 1.1 Sets . .7 1.2 Functions . .9 1.3 Fields . 11 1.4 Matrices . 14 1.5 Systems of Linear Equations . 16 2 Vector Spaces 21 2.1 Definition . 21 2.2 Linear Combinations, Span, and (Internal) Direct Sum . 24 2.3 Linear Independence and Bases . 27 2.4 Finitely Generated Vector Spaces and Dimension . 31 2.5 Existence of Bases . 35 3 Linear Transformations 39 3.1 Linear Transformations . 39 3.2 Kernel and Image . 41 3.3 Dual Spaces . 45 3.4 Coordinates . 47 4 Polynomials 51 4.1 The Algebra of Polynomials . 51 4.2 The Euclidean Algorithm . 54 4.3 Prime Factorization . 55 4.4 Irreducible Polynomials . 58 4.5 The Fundamental Theorem of Algebra . 59 4.6 The Polynomial Functional Calculus . 60 5 Eigenvalues, Eigenvectors, and the Spectrum 63 5.1 Eigenvalues and Eigenvectors . 63 5.2 Determinants . 67 5.3 The Characteristic Polynomial . 72 5.4 The Minimal Polynomial . 74 3 6 Operator Decompositions 77 6.1 Idempotents . 77 6.2 Matrix Decomposition . 80 6.3 Diagonalization . 81 6.4 Nilpotents . 83 6.5 Generalized Eigenvectors . 83 6.6 Primary Decomposition . 85 7 Canonical Forms 89 7.1 Cyclic Subspaces . 89 7.2 Rational Canonical Form . 92 7.3 The Cayley-Hamilton Theorem . 96 7.4 Jordan Canonical Form . 98 7.5 Uniqueness of Canonical Forms . 101 7.6 Holomorphic Functional Calculus . 106 8 Sesquilinear Forms and Inner Product Spaces 111 8.1 Sesquilinear Forms . 111 8.2 Inner Products and Norms . 114 8.3 Orthonormality . 116 8.4 Finite Dimensional Inner Product Subspaces . 119 9 Operators on Hilbert Space 123 9.1 Hilbert Spaces . 123 9.2 Adjoints . 124 9.3 Unitaries . 127 9.4 Projections . 129 9.5 Partial Isometries . 131 9.6 Dirac Notation and Rank One Operators . 133 10 The Spectral Theorems and the Functional Calculus 135 10.1 Spectra of Normal Operators . 135 10.2 Unitary Diagonalization . 136 10.3 The Spectral Theorems . 138 10.4 The Functional Calculus . 141 10.5 The Polar and Singular Value Decompositions . 146 4 Introduction These notes are designed to be a self contained treatment of linear algebra. The author assumes the reader has taken an elementary course in matrix theory and is well versed with matrices and Gaussian elimination. It is my intention that each time a definition is given, there will be at least two examples given. Most times, the examples will be presented without proof that they are, in fact, examples, and it will be left to the reader to verify that the examples set forth are examples. 5 6 Chapter 1 Background Material 1.1 Sets Throughout these notes, sets will be denoted with curly brackets or letters. A verti- cal bar in the middle of the curly brackets will mean \such that." For example, N = f1; 2; 3;::: g is the set of natural numbers, Z = f0; 1; −1; 2; −2;::: g is the set of integers, Q = p=q p; q 2 Z and q 6= 0 is the set of rational numbers, R is the set of real numbers 2 (which we will not define), and C = R + iR = x + iy x; y 2 R where i + 1 = 0 is the set of complex numbers. If z = x + iy is in C where x; y are in R, then its complex conjugate is the number z = x − iy. The modulus, or absolute value, of z is p p jzj = x2 + y2 = zz: Definition 1.1.1. To say x is an element of the set X, we write x 2 X. We say X is a subset of Y , denoted X ⊂ Y , if x 2 X ) x 2 Y (the double arrow ) means \implies"). Sets X and Y are equal if both X ⊂ Y and Y ⊂ X. If X ⊂ Y and we want to emphasize that X and Y may be equal, we will write X ⊆ Y . If X and Y are sets, we write (1) X [ Y = x x 2 X or x 2 Y , (2) X \ Y = x x 2 X and x 2 Y , (3) X n Y = x 2 X x2 = Y , and (4) X × Y = (x; y) x 2 X and y 2 Y . X × Y is called the (Cartesian) product of X and Y . There is a set with no elements in it, and it is denoted ; or fg. A subset X ⊂ Y is called proper if Y n X 6= ;, i.e., there is a y 2 Y n X. Remark 1.1.2. Note that sets can contain other sets. For example, fN; Z; Q; R; Cg is a set. In particular, there is a difference between f;g and ;. The first is the set containing the empty set. The second is the empty set. There is something in the first set, namely the empty set, so it is not empty. 7 Examples 1.1.3. (1) N [ Z = Z, N \ Z = N, and Z n N = f0; −1; −2;::: g. (2) ;[ X = X, ;\ X = ;, X n X = ;, and X n ; = X for all sets X. (3) R n Q is the set of irrational numbers. 2 (4) R × R = R = (x; y) x; y 2 R . Definition 1.1.4. If X is a set, then the power set of X, denoted P(X), is S S ⊂ X . Examples 1.1.5. (1) If X = ;, then P(X) = f;g. (2) If X = fxg, then P(X) = f;; fxgg. (3) If X = fx; yg, then P(X) = f;; fxg; fyg; fx; ygg. Exercises Exercise 1.1.6. A relation on a set X is a subset R of X × X. We usually write xRy if (x; y) 2 R. The relation R is called (i) reflexive if xRx for all x 2 X, (ii) symmetric if xRy implies yRx for all x; y 2 X, (iii) antisymmetric if xRy and yRx implies x = y, and (iv) transitive if xRy and yRz implies xRz. (v) skew-symmetric if xRy and xRz implies yRz for all x; y; z 2 X. Find the following examples of a relation R on a set X: 1. X and R such that R is transitive, but not symmetric, antisymmetric, or reflexive. 2. X and R such that R is reflexive, symmetric, and transitive, but not antisymmetric, 3. X and R such that R is reflexive, antisymmetric, and transitive, but not symmetric, and 4. X and R such that R is reflexive, symmetric, antisymmetric, and transitive. Exercise 1.1.7. Show that a reflexive relation R on X is symmetric and transitive if and only if R is skew-transitive. 8 1.2 Functions Definition 1.2.1. A function (or map) f from the set X to the set Y , denoted f : X ! Y , is a rule that associates to each x 2 X a unique y 2 Y , denoted f(x). The sets X and Y are called the domain and codomain of f respectively. The function f is called (1) injective (or one-to-one) if f(x) = f(y) implies x = y, (2) surjective (or onto) if for all y 2 Y there is an x 2 X such that f(x) = y, and (3) bijective if f is both injective and surjective. To say the element x 2 X maps to the element y 2 Y via f, i.e. f(x) = y, we sometimes write f : x 7! y or x 7! y when f is understood. The image, or range, of f, denoted im(f) or f(X), is y 2 Y there is an x 2 X with f(x) = y . Another way of saying that f is surjective is that im(f) = Y . The graph of f is the set (x; y) 2 X × Y y = f(x) . Examples 1.2.2. (1) The natural inclusion N ! Z is injective. (2) The absolute value function Z ! N [ f0g is surjective. (3) Neither of the first two is bijective. The map N ! Z given by 1 7! 0, 2 7! 1, 3 7! −1, 4 7! 2, 5 7! −2 and so forth is bijective. Definition 1.2.3. Suppose f : X ! Y and g : Y ! Z. The composite of g with f, denoted g ◦ f, is the function X ! Z given by (g ◦ f)(x) = g(f(x)). Examples 1.2.4. (1) (2) Proposition 1.2.5. Suppose f : X ! Y and g : Y ! Z. (1) If f; g are injective, then so is g ◦ f. (2) If f; g are surjective, then so is g ◦ f. Proof. Exercise. Proposition 1.2.6. Function composition is associative. Proof. Exercise. Definition 1.2.7. Let f : X ! Y , and let W ⊂ X. Then the restriction of f to W , denoted fjW : W ! Y , is the function given by fjW (w) = f(w) for all w 2 W . Examples 1.2.8. (1) (2) 9 Proposition 1.2.9. Let f : X ! Y . (1) f is injective if and only if it has a left inverse, i.e. a function g : Y ! X such that g ◦ f = idX , the identity on X. (2) f is surjective if and only if it has a right inverse, i.e. a function g : Y ! X such that f ◦ g = idY . Proof. (1) Suppose f is injective. Then for each y 2 im(f), there is a unique x with f(x) = y. Pick x0 2 X, and define a function g : Y ! X as follows: ( x if f(x) = y g(y) = x0 else.