(III.B) Linear Transformations in Terms of Matrices

(III.B) Linear transformations in terms of matrices

We have presented linear transformations in §III.A independently of matrices to emphasize the fact that, like vectors, they are intrinsic objects that exist independent of a basis. However, just as it is useful to write vectors ~v ∈ V in terms of their coordinates with respect to a given basis B = {~v1,...,~vn} for V , viz.   β1  .  [~v]B =  .  if ~v = β1~v1 + ... + βn~vn , βn it is also computationally convenient to summarize transformations in terms of their matrix of coef ficients with respect to two given bases. Namely, given T : V → W , and bases B = {~v1,...,~vn} , C = {w~ 1,..., w~ m} for V and W respectively, we write  β ··· β  11 1n m  . .. .  C [T]B =  . . .  if T~vj = ∑ βijw~ i (j = 1, . . . , n). i=1 βm1 ··· βmn Since   β1j  .  [T~vj]C =  .  , βmj we can rewrite this matrix  ↑ ↑    C [T]B =  [T~v1]C ··· [T~vn]C  . ↓ ↓

1 2 (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES

In this form clearly   0    .  ↑ ↑  .        th C [T]B[~vj]B =  [T~v1]C ··· [T~vn]C   1  ← j place   ↓ ↓  .   .  0

 ↑    =  [T~vj]C  , ↓ and so by linear extension to any ~v ∈ V

C [T]B[~v]B = [T~v]C.

An important special case is that of an endomorphism T : V → V where B = C . We will frequently write [T]B instead of B[T]B in this case.

d EXAMPLE 1. T = dx on V = P3 = polynomials of degree ≤ 3 , that is, d : P → P . dx 3 3 It sends 1 7→ 0, x 7→ 1, x2 7→ 2x, x3 7→ 3x2, so in terms of B = {1, x, x2, x3} we have     ↑ ↑ ↑ ↑ 0 1 0 0 d h i h i h i h i  0 0 2 0  =  d d d 2 d 3  =    dx 1 dx x dx x dx x    . dx B  B B B B   0 0 0 3  ↓ ↓ ↓ ↓ 0 0 0 0

Change of basis for transformations. Recall from §II.E that given 0 0 0 two bases A = {~u1,...,~u`}, A = { ~u1,..., ~u`} for a vector space U , we have PA→0A[~u]A = [~u]0A (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES 3 for all ~u ∈ U. To apply this to transformations, ﬁrst let’s consider a 0 change C → C : what is the new 0C [T]B , in terms of C [T]B ? Since

C [T]B[~v]B = [T~v]C = P0C→C · [T~v]0C , −1 (PC→0C · C [T]B) [~v]B = (P0C→C ) C [T]B[~v]B = [T~v]0C taking 0C [T]B to be PC→0C ·C [T]B gives

0C [T]B[~v]B = [T~v]0C , which is what we want. On the other hand, for B → 0B , writing

[T~v]C = C [T]B[~v]B = C [T]B (P0B→B · [~v]0B) , we see that writing C [T]0B = C [T]B P0B→B gives

C [T]0B[~v]0B = [T~v]C. So for B → 0B and C → 0C ,

0C [T]0B = (PC→0C ) ·C [T]B · (P0B→B) where e.g. PC→0C is the matrix with columns = the vectors of C written in terms of the 0C -basis. For T : V → V with C = B (coefﬁcients with respect to one basis), if we change B → 0B that means “on both sides”:

0B[T]0B = (PB→0B) ·B [T]B · (P0B→B) .

In words: “if we know [T]B then we can transform [~v]0B to [~v]B , 0 apply [T]B to get [T~v]B , and transform the result back to the B - basis to get [T~v]0B .” What we’ll usually know is P = P0B→B , i.e. the coordinates the vectors of a new basis 0B in terms of the old basis B (in terms of which we also have [T] ). (For instance, if B is the n standard basis of V = R , then P = P0B in the notation of §II.E.) In brief, then, −1 [T]0B = P [T]B P. The difﬁculty is always in getting the P ’s the right way around! 4 (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES

DEFINITION 2. Two m × m matrices M1 and M2 are said to be similar iff they are related by an invertible S via

−1 M1 = S M2S. It’s natural to think of similar matrices as “representing the same endomorphism with respect to different bases”.

h d i EXAMPLE 3. Let’s change dx from the previous example to h i B d where 0B = {1, x + t, (x + t)2, (x + t)3}. First of all we can dx 0 B h i write down d directly: dx 0B     0 1 0 0 ↑ ↑ ↑ ↑    h i h i h i h i   0 0 2 0   d 1 d (x + t) d (x + t)2 d (x + t)3  =    dx 0B dx 0B dx 0B dx 0B   0 0 0 3  ↓ ↓ ↓ ↓   0 0 0 0 h i which (surprise!) is the same matrix as d . Let’s see if the dx B −1 P [T]B P = [T]0B formula gives the same result. First we compute   1 t t2 t3  ↑ ↑ ↑ ↑   2  2 3  0 1 2t 3t  P0B→B =  [1] [x + t] (x + t) (x + t)  =   ,  B B B B   0 0 1 3t  ↓ ↓ ↓ ↓   0 0 0 1 and taking rre f (P0B→B |I4 ) to get (I4 |PB→0B ) gives  1 −t t2 −t3   − 2   0 1 2t 3t  PB→0B =   .  0 0 1 −3t  0 0 0 1 As you’ll readily verify, multiplying out the matrices d d PB→0B · · P0B→B to get dx B dx 0B h i h i does give the right answer (that d and d are the same ma- dx B dx 0B trix).

Since the last example (wow, a matrix similar to itself) could be considered somewhat disappointing, here is one more. (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES 5

EXAMPLE 4. We write the matrix (with respect to the standard basis) for the transformation T : R3 → R3 rotating thru an angle t θ about the axis spanned by ~v1 = (1, 2, 2). To do this, ﬁrst ex- pand this to an orthogonal basis B = {~v1,~v2,~v3} by choosing ~v2 = t t (−2, 2, −1) and ~v3 = (−2, −1, 2). (Note that these also all have the same length.) Then

     1 2 2  1 −2 −2 1 0 0 9 9 9 [T] = S[T] S−1 =  −   −   − 2 2 − 1  eˆ B  2 2 1   0 cos θ sin θ   9 9 9  2 1 2 2 −1 2 0 sin θ cos θ − 9 − 9 9   1 + 8 cos θ 2 − 2 cos θ − 6 sin θ 2 − 2 cos θ + 6 sin θ 1 =  2 − 2 cos θ + 6 sin θ 4 + 5 cos θ 4 − 4 cos θ − 3 sin θ  9   2 − 2 cos θ − 6 sin θ 4 − 4 cos θ + 3 sin θ 4 + 5 cos θ is a big ugly mess. But our technology made it easy to compute. ◦ In particular, rotation by 90 about eˆ1 resp. ~v1 are given by     1 0 0 1 −4 8 1 M =  0 0 −1  resp. M0 =  8 4 1  ,   9   0 1 0 −4 7 4 and we note that     1 −4 8 1 8 4 1 1 MM0 =  4 −7 −4  6=  8 1 −4  = M0 M. 9   9   8 4 1 −4 4 −7

This illustrates the fact that while rotations in R2 commmute, rotations in R3 do not unless the axes are the same.1

Rank + Nullity revisited. We have shown how to associate to any linear transformation T : V → W (plus two given bases) a matrix; conversely any matrix does give a linear transformation (with respect to the given bases). Due to this equivalence we should be able to write theorems about transformations as theorems about matrices. According to Exercise III.A.1 you can essentially think of V and W as Rn and Rm with the standard bases without any loss of

1You can even see this with 90◦ rotations about the x and y axes (try it with a sheet of paper!). 6 (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES generality; for simplicity we will now assume this setting: the “matrix B of T ” will just mean eˆ[T]eˆ . From this point of view I want to give a concrete proof (using rre f ) of rank+nullity. First note that all vectors “in the image of” T : Rn → Rm are linear combinations of the columns of B : so dim(im(T)) = dim(Vcol(B)) . On the other hand, dim(ker(T)) = dim{~v | B~v = 0}, where {~v | B~v = 0} is the subspace of Rn comprising solutions to the ho- mogeneous equation.

PROPOSITION 5. (Rank+Nullity, matrix version). If T : Rn → Rm has matrix B (in the standard basis) then

dim(Vcol(B)) + dim{~v | B~v = 0} = n. First we will prove a statement interesting in its own right, re- calling ﬁrst that rank(B) was deﬁned as the number of leading 1 s in rre f (B) , and rank(T) as the dimension of im(T).

LEMMA 6. rank(T) = dim(Vcol(B)) = rank(B) = dim(Vrow(B)).

PROOFOF LEMMA. The ﬁrst equality is clear, since the column space of B is exactly the image of T, writen with respect to the standard basis. We will need the following basic fact: the (left) action of an invertible m × m matrix P on vectors in Rm , preserves the dimensions of subspaces U ⊆ Rm . Here is one way of showing this:2 an invertible m × m matrix P gives rise to an invertible linear transformation P : Rm → Rm . If U ⊆ Rm is a subspace then the restriction of P to U

P|U : U → P(U) is still invertible (just restrict P −1 to P(U) to get the inverse transformation!). Since the range and domain of an invertible transformation have the same dimension (Prop. III.A.12),

dim(U) = dim(P(U)) = dim(P(U)).

2You could also prove this more “directly” (without tricks) by observing that any subspace U has a basis and considering the effect of an invertible P on that basis (show it takes it to a new basis, for P(U) ). Perhaps you should try this! (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES 7

So if U = Vcol(B) , and P = E is the invertible m × m matrix such that E· B = rre f (B) , then

dim(Vcol(B)) = dim(E(Vcol(B))) = dim(Vcol(rre f (B))). Clearly the columns of rre f (B) with leading 1’s are a basis for the column space of rre f (B) , so that dim(Vcol(rre f (B))) = # of leading 1’s= rank(B) . So dim(Vcol(B)) = rank(B). The fact that dim(Vrow(B)) = dim(Vrow(rre f (B))) = # of leading 1’s in rre f (B) comes from §§II.C-D.

PROOFOF PROPOSITION. Recall that in solving B~v = 0 , or equiv- alently rre f (B) = 0 , we were able to choose freely the coordinates xj (of ~v ) corresponding to columns not containing a leading 1. Now rre f (B) is an m × n matrix, so there are n columns and n − rank(B) columns without leading 1’s. So

dim{~v | B~v = 0} = n − rank(B).

But by the above Lemma, dim(Vcol(B)) = rank(B) . Add them. Wasn’t that easy?

Composition of linear transformations. The slogan here is: “matrix multiplication corresponds to composition of linear transformations”. To see this, consider T : V → W and U : W → Z, with bases A = {~v1,...,~vn} ⊂ V, B = {w~ 1,..., w~ m} ⊂ W, and C = {~z1,...,~zp}.

PROPOSITION 7. C [U ◦ T]A = C [U]B · B[T]A.

PROOF. For the matrices, write Q = C [U ◦ T]A, R = C [U]B, and m p S = B[T]A; this means that T~vj = ∑k=1 Skjw~ k, Uw~ k = ∑i=1 Rik~zi, and p (1) (U ◦ T)~vj = ∑ Qij~zi. i=1 8 (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES

But the LHS of (1) is the same as

m ! m m p U(T(~vj)) = U ∑ Skjw~ k = ∑ SkjUw~ k = ∑ Skj ∑ Rik~zi k=1 k=1 k=1 i=1

p m ! = ∑ ∑ RikSkj ~zi, i=1 k=1 and so p " m # ~ ∑ Qij − ∑ RikSkj ~zi = 0. i=1 k=1 m By independence of the {~zi}, we therefore have Qij = ∑k=1 RikSkj for each (i, j), which is exactly the statement that Q = R · S.

EXAMPLE 8. Recall that in the basis B = {1, x, x2, x3} of V = d P3(R), T = dx has matrix  0 1 0 0     0 0 2 0  M := [T]B =   .  0 0 0 3  0 0 0 0

By the Proposition, T ◦ T =: T2 resp. T ◦ T ◦ T =: T3 is represented by

 0 0 2 0   0 0 0 6      2  0 0 0 6  3  0 0 0 0  M =   resp. M =   ,  0 0 0 0   0 0 0 0  0 0 0 0 0 0 0 0 and3 Mk≥4 = 0 =⇒ Tk≥4 = 0. If (as here) some power of a matrix or transformation is zero, it is called nilpotent.

Recall that a transformation T is invertible (a 2-sided inverse exists) if and only if it is an isomorphism (1-1 and onto). With notation as above, Proposition 7 leads at once to the

3See the ﬁrst exercise below. (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES 9

COROLLARY 9. T : V → W is an invertible transformation ⇐⇒ −1 B[T]A is an invertible matrix. In this case, A[T ]B is given by the matrix −1 inverse (B[T]A) .

PROOF. If T is invertible , then we have S : W → V such that

ST = IdV and TS = IdW. But then I = A[IdV]A = A[S]B · B[T]A (and vice versa). If M = B[T]A is invertible, then by Exercise (1) below there exists −1 S with matrix A[S]B = M . So then the matrix of IdV − ST is the zero matrix, and applying this exercise again we see that IdV = ST. (Now reverse the order.)

More on change of basis. Up to this point I have separated the concepts of transformation and change of basis, although they are both represented by matrices. Here is how they are related, given two bases B, 0B for V:

PB→0B = 0B[I]B, where I : V → V is the identity transformation. This makes intuitive sense: like the identity, change of basis is not moving vectors. It merely writes them in different coordinates. In terms of our notational machinery for composing transformations, we have

0B[I]BB[T]00B = 0B[IT]00B = 0B[T]00B, which is to say that 0B[I]B does exactly what the change of basis matrix is supposed to do.

Some simple transformations in R2. To conclude our introduc- tion to linear transformations as it began, here are a few examples with geometric appeal: ! a −b • Any matrix of the form , considered as a tramsforma- b a tion of R2 written in the standard basis, is a “rotation-dilation”: in 10 (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES ! ! r 0 cos θ − sin θ fact, it is equal to the matrix product 0 r sin θ cos θ √ where θ = arctan(b/a) and r = a2 + b2 .

• A shear along the line L is a transformation T with (i) T(~v) = ~v (∀ ~v ∈ L ) and (ii) T(~x) −~x ∈ L (∀ ~x ∈ R2 ). In matrix form (w.r.t. eˆ ) ! 1 t the simplest examples are . Notice how these multiply: 0 1 ! ! ! 1 t 1 t 1 t + t 1 2 = 1 2 . 0 1 0 1 0 1

• The orthogonal projection4 to the line L (spanned by the unit vector uˆ) is given by the formula T(~α) = proj (~α) = (~α · uˆ)uˆ. It is important L ! 3 that uˆ be a unit vector; for instance, if L = span then you 4 ! 3/5 must ﬁrst normalize to get uˆ = . To ﬁnd the matrix [T]eˆ , 4/5 simply look at where T sends eˆ1 and eˆ2 ; these are then the columns of your matrix. For instance, ! !! ! ! ! 1 3/5 3/5 3 3/5 9/25 T(eˆ1) = · = = , 0 4/5 4/5 5 4/5 12/25 ! 12/25 T(eˆ2) = 16/25 and so   ↑ ↑ ! 1 9 12 [T]eˆ =  [Teˆ ]eˆ [Teˆ2]eˆ  = .  1  25 12 16 ↓ ↓

• To re f lect about the line L (spanned by uˆ) you just “overproject” as

4As with the example involving rotations in R3, we make informal ad hoc use of the dot product to obtain notions of length and angle. As remarked in §III.A, our vector spaces in general do not come with an inner product (a concept which we have yet to deﬁne). EXERCISES 11 in the following diagram:

proj( )− L

u^ refl ( ) L L

So you can “see” the formula

reﬂL(~α) = ~α + 2(projL(~α) −~α) = 2projL(~α) −~α = 2(~α · uˆ)uˆ −~α.

Now here’s a trick for finding the matrix of reflL in our above exam- ! 3 ple (uˆ = 1 ), since we already know the reflection matrix as 5 4 well as the matrix that takes~α to itself (the identity): just write ! ! ! 9/25 12/25 1 0 −7/25 24/25 2 − = . 12/25 16/25 0 1 24/25 7/25

Exercises (1) Given two vector spaces V, W over the same ﬁeld F, let L(V, W) denote the set of (F-)linear transformations from V to W. This is

itself a vector space! If C = {w~ 1,..., w~ m} and B = {~v1,...,~vn} are bases of W resp. V, then the “matrix realization” map

Φ : L(V, W) → Mm×n(F)

deﬁned by Φ(T) := C [T]B is itself a linear transformation. Prove that Φ is an isomorphism. (So in particular, if a matrix representing T is zero, then T is zero.) (2) Let V be a 2-dimensional vector space over the ﬁeld F, and let B be an ordered basis for V. If T is an endomorphism of V and ! a b [T]B = , c d 12 (III.B) LINEAR TRANSFORMATIONS IN TERMS OF MATRICES

prove that

T2 − (a + d)T + (ad − bc)I = 0,

where I : V → V is the identity. (3) Let θ be a real number. Prove that the matrices ! ! cos θ − sin θ eiθ 0 and sin θ cos θ 0 e−iθ are similar over the ﬁeld of complex numbers, by writing with respect to two distinct bases the transformation T : R2 → R2 given by rotation through the angle θ.

(4) Let (a, b) be a (constant) point in the xy-plane. Recall that P3(x, y) denotes real polynomials in 2 variables (of degree ≤ 3 ). Consider the linear transformation

H : P3(x, y) → M2×2(R) given by taking a polynomial f (x, y) to

 2 2  ∂ f ( ) ∂ f ( ) ∂x2 a, b ∂x∂y a, b H( f ) :=  2 2  . ∂ f ( ) ∂ f ( ) ∂x∂y a, b ∂y2 a, b (You can call this “the Hessian of f evaluated at (a, b) ” if you like.) Now recall that B = {1, x, y, xy, x2, y2, x2y, xy2, x3, y3} is a

basis of P3(x, y) (in that order) and ( ! ! ! !) 1 0 0 1 0 0 0 0 C = , , , 0 0 0 0 1 0 0 1

is a basis for M2×2(R) . (a) Write the matrix C [H]B . (First decide what its dimensions should be!) (b) What are the rank and nullity of H ? Check that these satisfy the rank-nullity theorem. Is H onto? 1-1?