Linear algebra Integer linear programming: formulations, techniques and applications. 03 Santiago Valdés Ravelo [email protected] August, 2020 “Algebra is the offer made by the devil to the mathematician. The devil says: I will give you this powerful machine, it will answer any question you like. All you need to do is give me your soul: give up geometry and you will have this marvellous machine.”1 Michael F. Atiyah. 1Michael F. Atiyah, “Special Article: Mathematics in the 20푡ℎ Century”, Bulletin of the London Mathematical Society, 2002, p. 7. 1 Vectors 2 Matrices 3 Linear systems 4 Exercises Vectors Vectors Definition Finite sequence of real values: ⎫ value 1 } } } value 2 } } } value 3 } 푛 + ⎬ 푣 ∈ ℝ , where 푛 ∈ ℕ … } } value 푛 − 1 } } } value 푛 } } ⎭ Vectors Special vectors. Null Vector with all the elements equal to zero (null): ⎫ 0 } } 0 } } 푣 = [0, 0, … , 0]푡 } … 푣 ⎬ } 0 } } } 0 } ⎭ Vectors Special vectors. Unit Vector with one element equals to 1 and the rest equal to 0: ⎫ 0 } } } … } } 0 } } 푣 = [0, … , 0, 1, 0, … , 0]푡 } 1 ⎬ 푣 } 0 } } } … } } 0 } } * unit vectors are those whose norm is equals to 1 ⎭ Vectors Operations. Sum Sum the elements at the same position: ⎫ } 푛 푣1 + 푤1 } 푣, 푤 ∈ ℝ ∶ } } 푣2 + 푤2 } 푣 + 푤 = [푣 , 푣 , … , 푣 ]푡 + [푤 , 푤 , … , 푤 ]푡 푣 + 푤 1 2 푛 1 2 푛 … ⎬ } } 푡 푣푛 + 푤푛 } 푣 + 푤 = [푣1 + 푤1, 푣2 + 푤2, … , 푣푛 + 푤푛] } } ⎭ Vectors Operations. Scalar multiplication Multiply each element of the vector by the scalar (real number): ⎫ } 푛 휆 × 푣1 } 휆 ∈ ℝ, 푣 ∈ ℝ ∶ } } 휆 × 푣2 } 휆 × 푣 = 휆 × [푣 , 푣 , … , 푣 ]푡 휆 × 푣 1 2 푛 … ⎬ } } 푡 휆 × 푣푛 } 휆 × 푣 = [휆 × 푣1, 휆 × 푣2, … , 휆 × 푣푛] } } ⎭ Vectors Operations. Internal product Multiply the elements at the same position and sum the multiplication results: 푣, 푤 ∈ ℝ푛 ∶ 푤1 ⎡ 푤 ⎤ 푣푡 × 푤 = [푣 , 푣 , … , 푣 ] × ⎢ 2 ⎥ 1 2 푛 ⎢ … ⎥ ⎣ 푤푛 ⎦ 푡 푛 푣 × 푤 = ∑푖=1 (푣푖 × 푤푖) * the result of the internal product is a scalar. Vectors Linear combination Given 푚 vectors: 푣1, 푣2, … , 푣푚 ∈ ℝ푛. A linear combination of them is: 푚 1 2 푚 푖 휆1 × 푣 + 휆2 × 푣 + … + 휆푚 × 푣 = ∑ (휆푖 × 푣 ). 푖=1 Where, 휆1, 휆2, …, 휆푚 are scalars: 휆1, 휆2, … , 휆푚 ∈ ℝ. Vectors Linear combination 1 2 푚 푚 푖 푣1 푣1 푣1 ∑푖=1 휆푖 × 푣1 1 2 푚 푚 푖 푣2 푣2 푣2 ∑푖=1 휆푖 × 푣2 휆1× + 휆2× + … + 휆푚× = … … … … 1 2 푚 푚 푖 푣푛 푣푛 푣푛 ∑푖=1 휆푖 × 푣푛 Vectors Affine combination If the sum of the scalars in a linear combination is equal to 1, then the combination is an AFFINE combination. Affine combinations are linear combinations where: 푚 ∑ 휆푖 = 1. 푖=1 Vectors Conical combination If the the scalars in a linear combination are non-negative, then the combination is a CONICAL combination. Conical combinations are linear combinations where: 휆푖 ∈ ℝ+, ∀푖 ∈ {1, 2, … , 푚}. Vectors Convex combination If a linear combination is affine and conical, then the combination is a CONVEX combination. Convex combinations are linear combinations where: 푚 ∑ 휆푖 = 1 푖=1 휆푖 ∈ ℝ+, ∀푖 ∈ {1, 2, … , 푚}. Vectors Linear subspace 푆 ⊆ ℝ푛 is a LINEAR SUBSPACE if for any two vectors 푣, 푤 ∈ 푆 and any two scalars 훼, 훽 ∈ ℝ, the vector resulting from the linear combination 훼 × 푣 + 훽 × 푤 is in 푆. Vectors Affine subspace 푆 ⊆ ℝ푛 is an AFFINE SUBSPACE if for any two vectors 푣, 푤 ∈ 푆 and any two scalars 훼, 훽 ∈ ℝ, where 훼 + 훽 = 1, the vector resulting from the affine combination 훼 × 푣 + 훽 × 푤 is in 푆. * Every linear subspace is an affine subspace. However, the opposite does not hold true. Vectors Conical set 푆 ⊆ ℝ푛 is a CONICAL SET if for any two vectors 푣, 푤 ∈ 푆 and any two non-negative scalars 훼, 훽 ∈ ℝ+, the vector resulting from the conical combination 훼 × 푣 + 훽 × 푤 is in 푆. * Every linear subspace is a conical set. However, the opposite does not hold true. Vectors Convex set 푆 ⊆ ℝ푛 is a CONVEX SET if for any two vectors 푣, 푤 ∈ 푆 and any two non-negative scalars 훼, 훽 ∈ ℝ+, where 훼 + 훽 = 1, the vector resulting from the convex combination 훼 × 푣 + 훽 × 푤 is in 푆. * Every conical, affine or linear subspace is a convex set. However, the opposite does not holdtrue. Vectors Linear independence The vectors 푣1, 푣2, … , 푣푚 ∈ ℝ푛 are LINEARLY INDEPENDENT if there exists only one linear combination resulting in the null vector, and is when all the scalars of the combination are equal to zero, i.e.: 푚 푖 푡 ∑ (휆푖 × 푣 ) = [0, 0, … , 0] ⇒ 휆푖 = 0, ∀푖 ∈ {1, 2, … , 푚} 푖=1 Vectors Affine independence The vectors 푣1, 푣2, … , 푣푚 ∈ ℝ푛 are AFFINELY INDEPENDENT if: 푚 푖 푡 ∑푖=1 (휆푖 × 푣 ) = [0, 0, … , 0] ⇒ 휆푖 = 0, ∀푖 ∈ {1, 2, … , 푚} 푚 ∑푖=1 휆푖 = 0 * Any set of linearly independent vectors is affine independent. However, the opposite does not hold true. Vectors Linear span (hull) The LINEAR SPAN (also called LINEAR HULL) of a set 푆 ⊆ ℝ푛 is the subspace containing all the linear combinations of the elements of 푆: 푚 푖 푖 푠푝푎푛(푆) = {∑ (휆푖 × 푣 ) | 푚 ∈ ℕ, 푣 ∈ 푆, 휆푖 ∈ ℝ} 푖=1 Vectors Affine span (hull) The AFFINE SPAN (also called AFFINE HULL) of a set 푆 ⊆ ℝ푛 is the subspace containing all the affine combinations of the elements of 푆: 푚 푚 푖 푖 푎푓푓(푆) = {∑ (휆푖 × 푣 ) | 푚 ∈ ℕ, 푣 ∈ 푆, 휆푖 ∈ ℝ, ∑ 휆푖 = 1} 푖=1 푖=0 Vectors Conical hull The CONICAL HULL of a set 푆 ⊆ ℝ푛 is the set containing all the conical combinations of the elements of 푆: 푚 푖 푖 푐표푛푒(푆) = {∑ (휆푖 × 푣 ) | 푚 ∈ ℕ, 푣 ∈ 푆, 휆푖 ∈ ℝ+} 푖=1 Vectors Convex hull The CONVEX HULL of a set 푆 ⊆ ℝ푛 is the set containing all the convex combinations of the elements of 푆: 푚 푚 푖 푖 푐표푛푣푒푥(푆) = {∑ (휆푖 × 푣 ) | 푚 ∈ ℕ, 푣 ∈ 푆, 휆푖 ∈ ℝ+, ∑ 휆푖 = 1} 푖=1 푖=0 Vectors Basis A set 푆 ⊂ ℝ푛 is a BASIS of ℝ푛, if the elements of 푆 are linearly independent and the linear span of 푆 is ℝ푛. Remark. If 푆 is a basis of ℝ푛, then for any vector 푣 ∈ ℝ푛, there exists exactly one linear combination of the vectors of 푆 equals to 푣. Vectors Change of basis Consider a basis 푆 of ℝ푛 and a vector 푣 ∈ ℝ푛. If the vector 푤 ∈ 푆 is multiplied by a scalar 휆 ≠ 0 in the linear combination of the elements of 푆 that results in 푣, then (푆 ⧵ {푤}) ∪ {푣} is a basis of ℝ푛. Matrices Matrices Definition Rectangular array of real values: ⎫ value 1, 1 value 1, 2 … value 1, 푛 } } } value 2, 1 value 2, 2 … value 2, 푛 } 푚×푛 + ⎬ 퐴푚×푛 ∈ ℝ ,(푚, 푛 ∈ ℕ ) … … … … } } value 푚, 1 value 푚, 2 … value 푚, 푛 } } ⎭ Matrices Rows 푛 푎푖. ∈ ℝ is the 푖-th row of of the matrix 퐴푚×푛 (1 ≤ 푖 ≤ 푚): value 1, 1 value 1, 2 … value 1, 푛 … … … … value 푖, 1 value 푖, 2 … value 푖, 푛 푎푖. → … … … … value 푚, 1 value 푚, 2 … value 푚, 푛 Matrices Columns 푚 푎.푗 ∈ ℝ is the 푗-th column of the matrix 퐴푚×푛 (1 ≤ 푗 ≤ 푛): 푎.푗 ↓ value 1, 1 … value 1, 푗 … value 1, 푛 value 2, 1 … value 2, 푗 … value 2, 푛 … … … … … value 푚, 1 … value 푚, 푗 … value 푚, 푛 Matrices Special matrices. Null Each column of the matrix is a null vector (i.e. all elements are equal to zero): 0 … 0 … 0 … … … … … 0 … 0 … 0 … … … … … 0 … 0 … 0 Matrices Special matrices. Identity Square matrix (푚 = 푛), where 푎.푗 is the unit vector where the value 1 occurs at position 푗 (1 ≤ 푗 ≤ 푛) (i.e. all elements are equal to zero except for the main diagonal, whose elements are equal to one): ⎫ 1 0 0 … 0 } } 0 1 0 … 0 } } } 0 0 ⋱ ⋱ … 퐼 ∈ ℝ푛×푛 ⎬ 푛×푛 ⋱ ⋱ } … … 0 } } } 0 0 … 0 1 } ⎭ Matrices Special matrices. Lower triangular A square matrix is LOWER TRIANGULAR if all the values above the main diagonal are equal to zero (i.e. if 푖 < 푗, then 푎푖,푗 = 0): 푎1,1 0 0 … 0 푎2,1 푎2,2 0 … 0 ⋱ 푎3,1 푎3,2 푎3,3 … … … ⋱ ⋱ 0 푎푛,1 푎푛,2 … 푎푛,푛−1 푎푛,푛 Matrices Special matrices. Upper triangular A square matrix is UPPER TRIANGULAR if all the values below the main diagonal are equal to zero (i.e. if 푖 > 푗, then 푎푖,푗 = 0): 푎1,1 푎1,2 푎1,3 … 푎1,푛 0 푎2,2 푎2,3 … 푎2,푛 ⋱ 0 0 푎3,3 … ⋱ ⋱ … … 푎푛−1,푛 0 0 … 0 푎푛,푛 Matrices Operations. Scalar multiplication Multiply each element of the matrix by the scalar: ⎫ } 휆 × 푎1,1 휆 × 푎1,2 … 휆 × 푎1,푛 } 푚×푛 } 휆 ∈ ℝ, 퐴푚×푛 ∈ ℝ ∶ } 휆 × 푎2,1 휆 × 푎2,2 … 휆 × 푎2,푛 } 휆 × 퐴 ∈ ℝ푚×푛 … … … … ⎬ } } 휆 × 푎푚,1휆 × 푎푚,2 … 휆×푎푚,푛 } } } ⎭ Matrices Operations. Scalar sum Sum each element of the matrix with the scalar: ⎫ } 휆 + 푎1,1 휆 + 푎1,2 … 휆 + 푎1,푛 } 푚×푛 } 휆 ∈ ℝ, 퐴푚×푛 ∈ ℝ ∶ } 휆 + 푎2,1 휆 + 푎2,2 … 휆 + 푎2,푛 } 휆 + 퐴 ∈ ℝ푚×푛 … … … … ⎬ } } 휆 + 푎푚,1휆 + 푎푚,2 … 휆+푎푚,푛 } } } ⎭ Matrices Operations. Sum Sum the elements at the same position (row and column): ⎫ } 푎1,1 + 푏1,1 푎1,2 + 푏1,2 … 푎1,푛 + 푏1,푛 } 푚×푛 }퐴푚×푛, 퐵푚×푛 ∈ ℝ ∶ } 푎2,1 + 푏2,1 푎2,2 + 푏2,2 … 푎2,푛 + 푏2,푛 } 퐴 + 퐵 ∈ ℝ푚×푛 … … … … ⎬ } } 푎푚,1 + 푏푚,1 푎푚,2 + 푏푚,2 … 푎푚,푛 + 푏푚,푛 } } } ⎭ Matrices Operations.