Linear Programming Lecture 1: Linear Algebra Review Lecture 1: Linear Algebra Review Linear Programming 1 / 24 1 Linear Algebra Review 2 Linear Algebra Review 3 Block Structured Matrices 4 Gaussian Elimination Matrices 5 Gauss-Jordan Elimination (Pivoting) Lecture 1: Linear Algebra Review Linear Programming 2 / 24 columns rows 2 3 a1• 6 a2• 7 = a a ::: a = 6 7 •1 •2 •n 6 . 7 4 . 5 am• 2 3 2 T 3 a11 a21 ::: am1 a•1 T 6 a12 a22 ::: am2 7 6 a•2 7 AT = 6 7 = 6 7 = aT aT ::: aT 6 . .. 7 6 . 7 1• 2• m• 4 . 5 4 . 5 T a1n a2n ::: amn a•n Matrices in Rm×n A 2 Rm×n 2 3 a11 a12 ::: a1n 6 a21 a22 ::: a2n 7 A = 6 7 6 . .. 7 4 . 5 am1 am2 ::: amn Lecture 1: Linear Algebra Review Linear Programming 3 / 24 rows 2 3 a1• 6 a2• 7 = 6 7 6 . 7 4 . 5 am• 2 3 2 T 3 a11 a21 ::: am1 a•1 T 6 a12 a22 ::: am2 7 6 a•2 7 AT = 6 7 = 6 7 = aT aT ::: aT 6 . .. 7 6 . 7 1• 2• m• 4 . 5 4 . 5 T a1n a2n ::: amn a•n Matrices in Rm×n A 2 Rm×n columns 2 3 a11 a12 ::: a1n 6 a21 a22 ::: a2n 7 A = 6 7 = a a ::: a 6 . .. 7 •1 •2 •n 4 . 5 am1 am2 ::: amn Lecture 1: Linear Algebra Review Linear Programming 3 / 24 2 3 2 T 3 a11 a21 ::: am1 a•1 T 6 a12 a22 ::: am2 7 6 a•2 7 AT = 6 7 = 6 7 = aT aT ::: aT 6 . .. 7 6 . 7 1• 2• m• 4 . 5 4 . 5 T a1n a2n ::: amn a•n Matrices in Rm×n A 2 Rm×n columns rows 2 3 2 3 a11 a12 ::: a1n a1• 6 a21 a22 ::: a2n 7 6 a2• 7 A = 6 7 = a a ::: a = 6 7 6 . .. 7 •1 •2 •n 6 . 7 4 . 5 4 . 5 am1 am2 ::: amn am• Lecture 1: Linear Algebra Review Linear Programming 3 / 24 2 T 3 a•1 T 6 a•2 7 = 6 7 = aT aT ::: aT 6 . 7 1• 2• m• 4 . 5 T a•n Matrices in Rm×n A 2 Rm×n columns rows 2 3 2 3 a11 a12 ::: a1n a1• 6 a21 a22 ::: a2n 7 6 a2• 7 A = 6 7 = a a ::: a = 6 7 6 . .. 7 •1 •2 •n 6 . 7 4 . 5 4 . 5 am1 am2 ::: amn am• 2 3 a11 a21 ::: am1 6 a12 a22 ::: am2 7 AT = 6 7 6 . .. 7 4 . 5 a1n a2n ::: amn Lecture 1: Linear Algebra Review Linear Programming 3 / 24 T T T = a1• a2• ::: am• Matrices in Rm×n A 2 Rm×n columns rows 2 3 2 3 a11 a12 ::: a1n a1• 6 a21 a22 ::: a2n 7 6 a2• 7 A = 6 7 = a a ::: a = 6 7 6 . .. 7 •1 •2 •n 6 . 7 4 . 5 4 . 5 am1 am2 ::: amn am• 2 3 2 T 3 a11 a21 ::: am1 a•1 T 6 a12 a22 ::: am2 7 6 a•2 7 AT = 6 7 = 6 7 6 . .. 7 6 . 7 4 . 5 4 . 5 T a1n a2n ::: amn a•n Lecture 1: Linear Algebra Review Linear Programming 3 / 24 Matrices in Rm×n A 2 Rm×n columns rows 2 3 2 3 a11 a12 ::: a1n a1• 6 a21 a22 ::: a2n 7 6 a2• 7 A = 6 7 = a a ::: a = 6 7 6 . .. 7 •1 •2 •n 6 . 7 4 . 5 4 . 5 am1 am2 ::: amn am• 2 3 2 T 3 a11 a21 ::: am1 a•1 T 6 a12 a22 ::: am2 7 6 a•2 7 AT = 6 7 = 6 7 = aT aT ::: aT 6 . .. 7 6 . 7 1• 2• m• 4 . 5 4 . 5 T a1n a2n ::: amn a•n Lecture 1: Linear Algebra Review Linear Programming 3 / 24 2 a11 3 2 a12 3 2 a1n 3 6 a21 7 6 a22 7 6 a2n 7 = x 6 7+ x 6 7+ ··· + x 6 7 1 6 . 7 2 6 . 7 n 6 . 7 4 . 5 4 . 5 4 . 5 am1 am2 amn = x1 a•1 + x2 a•2 + ··· + xn a•n A linear combination of the columns. Matrix Vector Multiplication A column space view of matrix vector multiplication. 2 a11 a12 ::: a1n 3 2 x1 3 6 a21 a22 ::: a2n 7 6 x2 7 6 7 6 7 6 . .. 7 6 . 7 4 . 5 4 . 5 am1 am2 ::: amn xn Lecture 1: Linear Algebra Review Linear Programming 4 / 24 2 a12 3 2 a1n 3 6 a22 7 6 a2n 7 + x 6 7+ ··· + x 6 7 2 6 . 7 n 6 . 7 4 . 5 4 . 5 am2 amn = x1 a•1 + x2 a•2 + ··· + xn a•n A linear combination of the columns. Matrix Vector Multiplication A column space view of matrix vector multiplication. 2 a11 a12 ::: a1n 3 2 x1 3 2 a11 3 6 a21 a22 ::: a2n 7 6 x2 7 6 a21 7 6 7 6 7 = x 6 7 6 . .. 7 6 . 7 1 6 . 7 4 . 5 4 . 5 4 . 5 am1 am2 ::: amn xn am1 Lecture 1: Linear Algebra Review Linear Programming 4 / 24 2 a1n 3 6 a2n 7 + ··· + x 6 7 n 6 . 7 4 . 5 amn = x1 a•1 + x2 a•2 + ··· + xn a•n A linear combination of the columns. Matrix Vector Multiplication A column space view of matrix vector multiplication. 2 a11 a12 ::: a1n 3 2 x1 3 2 a11 3 2 a12 3 6 a21 a22 ::: a2n 7 6 x2 7 6 a21 7 6 a22 7 6 7 6 7 = x 6 7+ x 6 7 6 . .. 7 6 . 7 1 6 . 7 2 6 . 7 4 . 5 4 . 5 4 . 5 4 . 5 am1 am2 ::: amn xn am1 am2 Lecture 1: Linear Algebra Review Linear Programming 4 / 24 = x1 a•1 + x2 a•2 + ··· + xn a•n A linear combination of the columns. Matrix Vector Multiplication A column space view of matrix vector multiplication. 2 a11 a12 ::: a1n 3 2 x1 3 2 a11 3 2 a12 3 2 a1n 3 6 a21 a22 ::: a2n 7 6 x2 7 6 a21 7 6 a22 7 6 a2n 7 6 7 6 7 = x 6 7+ x 6 7+ ··· + x 6 7 6 . .. 7 6 . 7 1 6 . 7 2 6 . 7 n 6 . 7 4 . 5 4 . 5 4 . 5 4 . 5 4 . 5 am1 am2 ::: amn xn am1 am2 amn Lecture 1: Linear Algebra Review Linear Programming 4 / 24 A linear combination of the columns. Matrix Vector Multiplication A column space view of matrix vector multiplication. 2 a11 a12 ::: a1n 3 2 x1 3 2 a11 3 2 a12 3 2 a1n 3 6 a21 a22 ::: a2n 7 6 x2 7 6 a21 7 6 a22 7 6 a2n 7 6 7 6 7 = x 6 7+ x 6 7+ ··· + x 6 7 6 . .. 7 6 . 7 1 6 . 7 2 6 . 7 n 6 . 7 4 . 5 4 . 5 4 . 5 4 . 5 4 . 5 am1 am2 ::: amn xn am1 am2 amn = x1 a•1 + x2 a•2 + ··· + xn a•n Lecture 1: Linear Algebra Review Linear Programming 4 / 24 Matrix Vector Multiplication A column space view of matrix vector multiplication. 2 a11 a12 ::: a1n 3 2 x1 3 2 a11 3 2 a12 3 2 a1n 3 6 a21 a22 ::: a2n 7 6 x2 7 6 a21 7 6 a22 7 6 a2n 7 6 7 6 7 = x 6 7+ x 6 7+ ··· + x 6 7 6 . .. 7 6 . 7 1 6 . 7 2 6 . 7 n 6 . 7 4 . 5 4 . 5 4 . 5 4 . 5 4 . 5 am1 am2 ::: amn xn am1 am2 amn = x1 a•1 + x2 a•2 + ··· + xn a•n A linear combination of the columns. Lecture 1: Linear Algebra Review Linear Programming 4 / 24 Range of A m n Ran (A) = fy 2 R j 9 x 2 R such that y = Ax g Ran (A) = the linear span of the columns of A The Range of a Matrix Let A 2 Rm×n (an m × n matrix having real entries). Lecture 1: Linear Algebra Review Linear Programming 5 / 24 Ran (A) = the linear span of the columns of A The Range of a Matrix Let A 2 Rm×n (an m × n matrix having real entries). Range of A m n Ran (A) = fy 2 R j 9 x 2 R such that y = Ax g Lecture 1: Linear Algebra Review Linear Programming 5 / 24 The Range of a Matrix Let A 2 Rm×n (an m × n matrix having real entries). Range of A m n Ran (A) = fy 2 R j 9 x 2 R such that y = Ax g Ran (A) = the linear span of the columns of A Lecture 1: Linear Algebra Review Linear Programming 5 / 24 The linear span of v1;:::; vk : Span [v1;:::; vk ] = fy j y = ξ1v1 + ξ2v2 + ··· + ξk vk ; ξ1; : : : ; ξk 2 Rg The subspace orthogonal to v1;:::; vk : ? n fv1;:::; vk g = fz 2 R j z • vi = 0; i = 1;:::; k g ? ? Facts: fv1;:::; vk g = Span [v1;:::; vk ] ? h ?i Span [v1;:::; vk ] = Span [v1;:::; vk ] Two Special Subspaces n Let v1;:::; vk 2 R . Lecture 1: Linear Algebra Review Linear Programming 6 / 24 The subspace orthogonal to v1;:::; vk : ? n fv1;:::; vk g = fz 2 R j z • vi = 0; i = 1;:::; k g ? ? Facts: fv1;:::; vk g = Span [v1;:::; vk ] ? h ?i Span [v1;:::; vk ] = Span [v1;:::; vk ] Two Special Subspaces n Let v1;:::; vk 2 R .