Linear Algebra with Sagemath∗

Ajit Kumar ICT Mumbai [email protected] July 11, 2019

1 Working with Vectors

Vectors in Sage can be deﬁned by vector function and giving the entries in a list. One can also mention the domain from which entries are taken. Suppose we want a vector v ∈ Q3, then we deﬁne as v=vector(QQ,[1/2,1,3]). Let us explore some inbuilt methods on vectors in SageMath. sage: v1 = vector(QQ,[1,-1,2]) sage: v2 = vector(QQ,[2,3,-1]) sage: 2*v1-3*v2# linear combination (-4, -11, 7) sage: v1.norm() sqrt (6) sage: v1.dot_product(v2)# Gives the dot product of v1 and v2 -3 sage: v1.cross_product(v2)# Gives the dot product of v1 and v2 (-5, 5, 5)

Exercise 1.1. Check that for any two vectors u and v in R3, u × v is orthogonal to u and v. Example 1. Find the orthogonal projection of v = (3, 2, 5) onto (1, −2, 1). sage: v = vector([3,2,5]) sage: u = vector([1,-2,1]) sage: p = v.dot_product(u)/(norm(u)^2)*u sage: p (2/3, -4/3, 2/3) Check that u is orthogonal to v − p. sage: u.dot_product(v-p) 0 We can create a SageMath subroutine to ﬁnd the orthogonal projection of v onto u. sage: def proj(v,u): ....: a = v.dot_product(u)/(norm(u)^2)*u ....: return(a) sage: u1 = vector([3,-2,4]) sage: u2 = vector([2,1,3]) sage: proj(u2,u1) (48/29, -32/29, 64/29)

∗Distributed during “Teachers Enrichment Course in Undergraduate Mathematics Curriculum” at IIT Guwahati during July 1-13, 2019

1 2 System of Linear Equations

We can solve a system of linear equations in SageMath in many ways. sage: x,y,z=var(’x,y,z’)# To declarea,y,z ar variables sage: solve([x-2*y+z==10, 2*y+3*z==-3,9*x+3*y+z==2],[x,y,z]) [ [x == (101/79), y == (-288/79), z == (113/79)] ] We can write the above system as a matrix equation Ax = b and solve. sage: A = matrix(RR,[[1,-2,1],[0,2,3],[9,3,1]]) sage: b = vector(RR,[10,-3,2]) sage: A.solve_right(b) (1.27848101265823, -3.64556962025316, 1.43037974683544) sage: A\b# MatLab like command (1.27848101265823, -3.64556962025316, 1.43037974683544) sage: aug = A.augment(b,subdivide=True) sage: aug [ 1.00000000000000 -2.00000000000000 1.00000000000000| 10.0000000000000] [0.000000000000000 2.00000000000000 3.00000000000000|-3.00000000000000] [ 9.00000000000000 3.00000000000000 1.00000000000000| 2.00000000000000] sage: A1= aug.rref() sage: A1.column(3) (1.27848101265823, -3.64556962025316, 1.43037974683544)

2.1 Cramer’s Rule Example 2. Solve the Ax = b using the Cramer’s Rule where

 1 1 2 2   9   2 −3 4 −3   1  A =   , b =    3 7 6 −5   10  8 6 4 −3 5

First of all let us deﬁne a Sage subroutine in which we can replace any column of a matrix by appropriate vector. sage: def column_replace(M,col,v): ....: n1=M.nrows() ....: n2=M.ncols() ....: S=matrix(n1,n2) ....: fori in range(n1): ....: forj in range(n2): ....: S[i,j]=M[i,j] ....: fori in range(n1): ....: S[i,col-1]=v[i,0] ....: returnS sage: A=matrix([[1,1,2,2],[2,-3,4,-3],[3,7, 6,-5],[8,6,4,-3]]) sage: b=matrix(4,1,[9,1,10,5]) sage: A1=column_replace(A,1,b) sage: A2=column_replace(A,2,b)

2 sage: A3=column_replace(A,3,b) sage: A4=column_replace(A,4,b) sage: sol = matrix(4,1,[det(A1)/det(A), det(A2)/det(A), det(A3)/det (A), det(A4)/det(A)]) sage: sol [ -140/249] [ 68/83] [1259/498] [ 153/83]

3 Vector Spaces

Simplest way to define a vector space in Sage is to begin with the field and use an exponent to indicate the dimension. For example QQ^4 is a vector space of dimension 4 over the field Q. Similarly RR^4, CC^4 are vectors spaces of dimension 4 over R and C.

4 Linear spans and all that

Example 3. Find the linear span of the set of vectors

{(1, 1, 2, −1), (2, 3, 5, −4), (3, 4, 7, −5)}. sage: V = QQ^4 sage: v1 = vector(QQ, [1,1,2,-1]) sage: v2 = vector(QQ, [2,3,5,-4]) sage: v3 = vector(QQ, [3,4,7,-5]) sage: W = V.span([v1, v2,v3]) sage:W Vector space of degree 4 and dimension 2 over Rational Field Basis matrix: [ 1 0 1 1] [ 0 1 1 -2] Example 4. Check if the following set of vectors in Q4 are linearly dependent?

{(1, −1, 2, 4), (−3, 3, 2, 1), (−1, −2, 6, 9)} sage: V = QQ^4 sage: v1 = vector(QQ, [ 1, -1, 2, 4]) sage: v2 = vector(QQ, [-3, 3, 2, 1]) sage: v3 = vector(QQ, [-1, -2, 6, 9]) sage: L = [v1, v2, v3] sage: V.linear_dependence(L) == [] True

Example 5. Show that the following set of vectors in Q4 is a basis of Q4. B = {(2, −1, 3, 1), (3, 3, 4, 0), (1, −2, 0, −1), (4, 5, 5, −1)}

Hence ﬁnd the coordinates of w = (2, 3, 5, 9) with respect to B. sage: V=QQ^4 sage: v1 = vector(QQ, [2, -1, 3, 1])

3 sage: v2 = vector(QQ, [3, 3, 4, 0]) sage: v3 = vector(QQ, [1, -2, 0, -1]) sage: v4 = vector(QQ, [4, 5, 5, -1]) sage: L = [v1,v2,v3,v4] sage: V.linear_dependence(L) == [] True sage: w = vector(QQ,[2,3,5,9]) sage: W = V.subspace_with_basis(L) sage: s = W.coordinates(w) sage: s [-19/3, 26, 2/3, -16] sage: sum([s[i]*L[i] fori in range(4)])==w True We can obtain the coordinates of w with respect the above basis using RREF as follows: sage: M = column_matrix([v1,v2,v3,v4,w]) sage: M1 = M.rref() sage: M1.column(4) (-19/3, 26, 2/3, -16) Example 6. Complete the following linearly independent set of vectors {(2, 0, 2, 0), (1, 0, 2, 0)} in Q4 to a basis of Q4. sage: v1 = vector(QQ, [2,0,2,0]) sage: v2 = vector(QQ, [1,0,3,0]) sage: M = column_matrix([v1,v2]) sage: I4 = identity_matrix(4) sage: M1=M.augment(I4) sage: M1.rref() [ 1 0 3/4 0-1/4 0] [ 0 1-1/2 0 1/2 0] [ 0 0 0 1 0 0] [ 0 0 0 0 0 1] sage: [M1.column(p) forp in M1.pivots()] [(2, 0, 2, 0), (1, 0, 3, 0), (0, 1, 0, 0), (0, 0, 0, 1)]

5 Matrix Space

The four fundamental subspaces associated to an m × n matrix A are:

• The column space, col(A), spanned by all of the columns of A. This is a subspace of Rm. • The row space, row(A), spanned by all of the rows of A. This is a subspace of Rn. This also happens to be the column space of AT . • The nullspace (or kernel),null(A), consisting of all those vectors x for which Ax = 0. This is a subspace of Rn. • The left nullspace, which is just the nullspace of AT . This is a subspace of Rm.  1 3 −2 0 2 0   2 6 −5 −2 4 −3  Example 7. Consider a matrix B =  . Find the four fundamental  0 0 5 10 0 15  2 6 0 8 4 18 subspaces associated to B.

4 sage: B = matrix([[1, 3,-2 ,0 ,2 , 0],[2,6 ,-5,-2,4,-3],[0,0,5,10,0,15],[2,6,0,8,4,18]]) sage: B.column_space() Free module of degree 4 and rank 3 over Integer Ring Echelon basis matrix: [ 1 0 10 4] [ 0 1 -5 2] [ 0 0 0 6] sage: B.row_space() Free module of degree 6 and rank 3 over Integer Ring Echelon basis matrix: [1 3 0 4 2 0] [0 0 1 2 0 3] [0 0 0 0 0 6] sage: B.right_kernel()# Null space ofB Free module of degree 6 and rank 3 over Integer Ring Echelon basis matrix: [ 1 1 0 0 -2 0] [ 0 2 0 0 -3 0] [ 0 0 2 -1 2 0] sage: B.left_kernel() Free module of degree 4 and rank 1 over Integer Ring Echelon basis matrix: [10 -5 -1 0] sage: (B.T).right_kernel() Free module of degree 4 and rank 1 over Integer Ring Echelon basis matrix: [10 -5 -1 0] Exercise 5.1. Find the four fundamental subspaces associated to B using rref.

6 Gram-Schmidt Process

Example 8. Apply the Gram-Schmidt orthogonalization process ﬁnd an orthogonal basis from a basis Show that the following set of vectors in Q4 is a basis of Q4. L = {(2, −1, 3, 1), (3, 3, 4, 0), (1, −2, 0, −1), (4, 5, 5, −1)} of R4. sage: v1 = vector([2, -1, 3, 1]) sage: v2 = vector([3, 3, 4, 0]) sage: v3 = vector([1, -2, 0, -1]) sage: v4 = vector([4, 5, 5, -1]) sage: L = [v1,v2,v3,v4] We shall use the orthogonal function we have deﬁned earlier. sage: u1 = v1/norm(v1).n() sage: q2 = v2 - proj(v2,u1) sage: u2 = q2/norm(q2).n() sage: q3 = v3 - proj(v3,u1)-proj(v3,u2) sage: u3 = q3/norm(q3).n() sage: q4 = v4 - proj(v4,u1)-proj(v4,u2)-proj(v4,u3) sage: u4 = q4/norm(q4).n() sage: G = [u1,u2,u3,u4]

5 sage:G [(0.516397779494322, -0.258198889747161, 0.774596669241483, 0.258198889747161), (0.229415733870562, 0.917662935482247, 0.229415733870562, -0.229415733870562), (0.489142426032054, -0.286738663536032, -0.151802821872017, -0.809615049984090), (-0.664410597026756, -0.0949157995752659, 0.569494797451495, -0.474578997876241)] Next we shall write a SageMath subroutine to ﬁnd an orthonormal basis. sage: def Gram_Schmidt(S): ....: n = len(S) ....: E=[S[0]/norm(S[0]).n()] ....: fork in range(1,n): ....: q=S[k]-sum([proj(S[k],E[i]) fori in range(k)]) ....: E.append(q/norm(q).n()) ....: return(E) Now let us call Gram_Schmidt to ﬁnd an orthonormal basis from the L = {v1, v2, v3, v4} in the example 8. sage: G = Gram_Schmidt(L) sage:G [(0.516397779494322, -0.258198889747161, 0.774596669241483, 0.258198889747161), (0.229415733870562, 0.917662935482247, 0.229415733870562, -0.229415733870562), (0.489142426032054, -0.286738663536032, -0.151802821872017, -0.809615049984090), (-0.664410597026756, -0.0949157995752628, 0.569494797451498, -0.474578997876240)]

7 Linear Transformation

SageMath function linear_transformation deﬁnes a linear transformation between two vectors spaces in various forms. Readers may look the help document using linear_transformation? to see the options available to deﬁne linear transformations

Example 9. Deﬁne a linear transformation T (x1, x2, x3) = (x1−x2−x3, x1+3x2+x3, −3x1+x2−x3) from Q3 to Q3. Hence ﬁnd the image and kernel of T . Check if T is one-one, onto or bijection. sage: V = W = QQ^3 sage: var(’x1,x2,x3’) (x1 , x2 , x3) sage: f(x1,x2,x3)=[x1-x2-x3, x1+3*x2+x3,-3*x1+x2-x3] sage: T = linear_transformation(V, W, f) sage:T Vector space morphism represented by the matrix: [ 1 1 -3] [-1 3 1] [-1 1 -1] Domain: Vector space of dimension 3 over Rational Field Codomain: Vector space of dimension 3 over Rational Field The image of any vector u can be obtained by T(u). sage: u=vector(QQ, [1,2,-1]) sage: T(u) (0, 6, 0)

6 sage: T.image() Vector space of degree 3 and dimension 3 over Rational Field Basis matrix: [1 0 0] [0 1 0] [0 0 1] sage: T.kernel() Vector space of degree 3 and dimension 0 over Rational Field Basis matrix: [] sage: T.is_injective() True sage: T.is_surjective() True sage: T.is_bijective() True Find the matrix of T with respect to the standard basis on domain and co-domain. sage: A = T.matrix(side=’right’) sage:A [ 1 -1 -1] [ 1 3 1] [-3 1 -1] Let B = {(1, 2, −1) , (−2, 1, 4) , (5, 7, 2)} be a basis of domain of T and C = {(2, −1, 3) , (−2, 1, 4) , (5, 2, 4)}, a basis of the co-domain of T . Find the matrix of T with respect to basis B and C. sage: u1=vector(QQ, [1,2,-1]) sage: u2=vector(QQ,[-2,1,4]) sage: u3=vector(QQ, [5,7,2]) sage: B = [u1,u2,u3] sage: V.linear_dependence(B)==[] True sage: v1=vector(QQ, [2,-1,3]) sage: v2=vector(QQ,[-2,1,4]) sage: v3=vector(QQ, [5,2,4]) sage: C = [v1,v2,v3] sage: W.linear_dependence(C)==[] True We can ﬁnd the matrix of T with respect to the bases B and C using rref as follows: sage: M = column_matrix([v1,v2,v3,T(u1),T(u2),T(u3)]).rref() sage:M [ 1 0 0 -8/3 -47/21-890/63] [ 0 1 0 2/3 44/21146/63] [ 0 0 1 4/3 1/3 52/9] sage: M[:,3:6] [ -8/3 -47/21 -890/63] [ 2/3 44/21 146/63] [ 4/3 1/3 52/9] We can also ﬁnd the matrix of T with respect to the bases B and C using the inbuilt SageMath function. sage: V1 = V.subspace_with_basis(B) sage: W1 =W.subspace_with_basis(C)

7 sage: S = T.restrict_domain(V1).restrict_codomain(W1) sage: S.matrix(side=’right’) [ -8/3 -47/21 -890/63] [ 2/3 44/21 146/63] [ 4/3 1/3 52/9] We can ﬁnd the eigenvalues of the linear transformation by SageMath method T.eigenvalues(). Let us show that eigenvalues of T is same as eigenvalues of the matrix of T with respect to the standard basis. sage: T.eigenvalues() [3, 2, -2] sage: A.eigenvalues() [3, 2, -2] Let TB be the matrix of T with respect to the basis B deﬁned earlier of both domain and co-domain of T . Then we can show that eigenvalues of TB is same as eigenvalues of T . This shows that eigenvalues of a linear transformation is independent of basis. sage: V1 = V.subspace_with_basis(B) sage: TB = T.restrict_domain(V1).restrict_codomain(V1) sage: TB.eigenvalues() [3, 2, -2]

8 Eigenvalues and Eigenvectors

 1 −1 −1  Example 10. Consider the matrix A =  1 3 1 . (i) Find the characteristics polyno- −3 1 −1 mial, p(x), of A. (ii) Find the roots of p(x), show that eigenvalues of A are roots of p(x). (iii) Find the eigenvectors of A. (iv) Show that sum of the eigenvalues of A is trace of A. (iv) Show that the product of eigenvalues of A is the determinant of A. sage: A =matrix([[1, -1, -1],[1, 3, 1],[-3, 1, -1]]) sage: p(x) = A.characteristic_polynomial(x) sage: p(x) x^3 - 3*x^2 - 4*x + 12 sage: ev = A.eigenvalues() sage: ev [3, 2, -2] sage: p.roots() [(-2, 1), (2, 1), (3, 1)] sage: A.eigenvectors_right() [(3 , [ (1, -1, -1) ], 1) , (2, [ (1, 0, -1) ], 1) , (-2, [ (1, -1, 4) ], 1)] sage: sum(ev)==A.trace() True sage: prod(ev)==A.det() True Let us verify the Cayley-Hamilton theorem for A.

8 sage: coeff=p(x).coeffs() sage: coeff [[12, 0], [-4, 1], [-3, 2], [1, 3]] sage: sum([coeff[k][0]*A^(coeff[k][1]) fork in range(4)]) [0 0 0] [0 0 0] [0 0 0] Let P be the matrix of eigenvectors of A. Show that P −1 ∗ AP is a diagonal matrix whose diagonal entries are eigenvalues. sage: ev, P = A.eigenmatrix_right() sage: P.inverse()*A*P [ 3 0 0] [ 0 2 0] [ 0 0 -2]