MATH 532: Linear Algebra Chapter 4: Vector Spaces Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2015 [email protected] MATH 532 1 Outline 1 Spaces and Subspaces 2 Four Fundamental Subspaces 3 Linear Independence 4 Bases and Dimension 5 More About Rank 6 Classical Least Squares 7 Kriging as best linear unbiased predictor [email protected] MATH 532 2 Spaces and Subspaces Outline 1 Spaces and Subspaces 2 Four Fundamental Subspaces 3 Linear Independence 4 Bases and Dimension 5 More About Rank 6 Classical Least Squares 7 Kriging as best linear unbiased predictor [email protected] MATH 532 3 Spaces and Subspaces Spaces and Subspaces While the discussion of vector spaces can be rather dry and abstract, they are an essential tool for describing the world we work in, and to understand many practically relevant consequences. After all, linear algebra is pretty much the workhorse of modern applied mathematics. Moreover, many concepts we discuss now for traditional “vectors” apply also to vector spaces of functions, which form the foundation of functional analysis. [email protected] MATH 532 4 Spaces and Subspaces Vector Space Definition A set V of elements (vectors) is called a vector space (or linear space) over the scalar field F if (A1) x + y 2 V for any x; y 2 V (M1) αx 2 V for every α 2 F and (closed under addition), x 2 V (closed under scalar (A2) (x + y) + z = x + (y + z) for all multiplication), x; y; z 2 V, (M2) (αβ)x = α(βx) for all αβ 2 F, (A3) x + y = y + x for all x; y 2 V, x 2 V, (A4) There exists a zero vector 0 2 V (M3) α(x + y) = αx + αy for all α 2 F, such that x + 0 = x for every x; y 2 V, x 2 V, (M4) (α + β)x = αx + βx for all (A5) For every x 2 V there is a α; β 2 F, x 2 V, negative (−x) 2 V such that (M5)1 x = x for all x 2 V. x + (−x) = 0, [email protected] MATH 532 5 V = Rm and F = R (traditional real vectors) V = Cm and F = C (traditional complex vectors) × V = Rm n and F = R (real matrices) × V = Cm n and F = C (complex matrices) But also V is polynomials of a certain degree with real coefficients, F = R V is continuous functions on an interval [a; b], F = R Spaces and Subspaces Examples of vector spaces [email protected] MATH 532 6 V = Cm and F = C (traditional complex vectors) × V = Rm n and F = R (real matrices) × V = Cm n and F = C (complex matrices) But also V is polynomials of a certain degree with real coefficients, F = R V is continuous functions on an interval [a; b], F = R Spaces and Subspaces Examples of vector spaces V = Rm and F = R (traditional real vectors) [email protected] MATH 532 6 × V = Rm n and F = R (real matrices) × V = Cm n and F = C (complex matrices) But also V is polynomials of a certain degree with real coefficients, F = R V is continuous functions on an interval [a; b], F = R Spaces and Subspaces Examples of vector spaces V = Rm and F = R (traditional real vectors) V = Cm and F = C (traditional complex vectors) [email protected] MATH 532 6 But also V is polynomials of a certain degree with real coefficients, F = R V is continuous functions on an interval [a; b], F = R Spaces and Subspaces Examples of vector spaces V = Rm and F = R (traditional real vectors) V = Cm and F = C (traditional complex vectors) × V = Rm n and F = R (real matrices) × V = Cm n and F = C (complex matrices) [email protected] MATH 532 6 Spaces and Subspaces Examples of vector spaces V = Rm and F = R (traditional real vectors) V = Cm and F = C (traditional complex vectors) × V = Rm n and F = R (real matrices) × V = Cm n and F = C (complex matrices) But also V is polynomials of a certain degree with real coefficients, F = R V is continuous functions on an interval [a; b], F = R [email protected] MATH 532 6 Q: What is the difference between a subset and a subspace? A: The structure provided by the axioms (A1)–(A5), (M1)–(M5) Theorem The subset S ⊆ V is a subspace of V if and only if αx + βy 2 S for all x; y 2 S; α; β 2 F: (1) Remark Z = f0g is called the trivial subspace. Spaces and Subspaces Subspaces Definition Let S be a nonempty subset of V. If S is a vector space, then S is called a subspace of V. [email protected] MATH 532 7 A: The structure provided by the axioms (A1)–(A5), (M1)–(M5) Theorem The subset S ⊆ V is a subspace of V if and only if αx + βy 2 S for all x; y 2 S; α; β 2 F: (1) Remark Z = f0g is called the trivial subspace. Spaces and Subspaces Subspaces Definition Let S be a nonempty subset of V. If S is a vector space, then S is called a subspace of V. Q: What is the difference between a subset and a subspace? [email protected] MATH 532 7 Theorem The subset S ⊆ V is a subspace of V if and only if αx + βy 2 S for all x; y 2 S; α; β 2 F: (1) Remark Z = f0g is called the trivial subspace. Spaces and Subspaces Subspaces Definition Let S be a nonempty subset of V. If S is a vector space, then S is called a subspace of V. Q: What is the difference between a subset and a subspace? A: The structure provided by the axioms (A1)–(A5), (M1)–(M5) [email protected] MATH 532 7 Remark Z = f0g is called the trivial subspace. Spaces and Subspaces Subspaces Definition Let S be a nonempty subset of V. If S is a vector space, then S is called a subspace of V. Q: What is the difference between a subset and a subspace? A: The structure provided by the axioms (A1)–(A5), (M1)–(M5) Theorem The subset S ⊆ V is a subspace of V if and only if αx + βy 2 S for all x; y 2 S; α; β 2 F: (1) [email protected] MATH 532 7 Spaces and Subspaces Subspaces Definition Let S be a nonempty subset of V. If S is a vector space, then S is called a subspace of V. Q: What is the difference between a subset and a subspace? A: The structure provided by the axioms (A1)–(A5), (M1)–(M5) Theorem The subset S ⊆ V is a subspace of V if and only if αx + βy 2 S for all x; y 2 S; α; β 2 F: (1) Remark Z = f0g is called the trivial subspace. [email protected] MATH 532 7 “(=”: Only (A1), (A4), (A5) and (M1) need to be checked (why?). In fact, we see that (A1) and (M1) imply (A4) and (A5): If x 2 S, then — using (M1) — −1x = −x 2 S, i.e., (A5) holds. Using (A1), x + (−x) = 0 2 S, so that (A4) holds. Spaces and Subspaces Proof. “=)”: Clear, since we actually have (1) () (A1) and (M1) [email protected] MATH 532 8 In fact, we see that (A1) and (M1) imply (A4) and (A5): If x 2 S, then — using (M1) — −1x = −x 2 S, i.e., (A5) holds. Using (A1), x + (−x) = 0 2 S, so that (A4) holds. Spaces and Subspaces Proof. “=)”: Clear, since we actually have (1) () (A1) and (M1) “(=”: Only (A1), (A4), (A5) and (M1) need to be checked (why?). [email protected] MATH 532 8 If x 2 S, then — using (M1) — −1x = −x 2 S, i.e., (A5) holds. Using (A1), x + (−x) = 0 2 S, so that (A4) holds. Spaces and Subspaces Proof. “=)”: Clear, since we actually have (1) () (A1) and (M1) “(=”: Only (A1), (A4), (A5) and (M1) need to be checked (why?). In fact, we see that (A1) and (M1) imply (A4) and (A5): [email protected] MATH 532 8 Using (A1), x + (−x) = 0 2 S, so that (A4) holds. Spaces and Subspaces Proof. “=)”: Clear, since we actually have (1) () (A1) and (M1) “(=”: Only (A1), (A4), (A5) and (M1) need to be checked (why?). In fact, we see that (A1) and (M1) imply (A4) and (A5): If x 2 S, then — using (M1) — −1x = −x 2 S, i.e., (A5) holds. [email protected] MATH 532 8 Spaces and Subspaces Proof. “=)”: Clear, since we actually have (1) () (A1) and (M1) “(=”: Only (A1), (A4), (A5) and (M1) need to be checked (why?). In fact, we see that (A1) and (M1) imply (A4) and (A5): If x 2 S, then — using (M1) — −1x = −x 2 S, i.e., (A5) holds. Using (A1), x + (−x) = 0 2 S, so that (A4) holds. [email protected] MATH 532 8 Remark span(S) contains all possible linear combinations of vectors in S. One can easily show that span(S) is a subspace of V. Example (Geometric interpretation) 3 1 If S = fv 1g ⊆ R , then span(S) is the line through the origin with direction v 1. 3 2 If S = fv 1; v 2 : v 1 6= αv 2; α 6= 0g ⊆ R , then span(S) is the plane through the origin “spanned by” v 1 and v 2. Spaces and Subspaces Definition Let S = fv 1;:::; v r g ⊆ V.