NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS

LANCE D. DRAGER

1. Introduction A problem that comes up in a lot of different fields of mathematics and engineer- ing is solving a system of linear constant coefficient differential equations. Such a system looks like

′ 푥1(푡) = 푎11푥1(푡) + 푎12푥2(푡) + ··· + 푎1푛푥푛(푡) ′ 푥2(푡) = 푎21푥1(푡) + 푎22푥2(푡) + ··· + 푎2푛푥푛(푡) (1.1) . . ′ 푥푛(푡) = 푎푛1푥1(푡) + 푎푛2푥2(푡) + ··· + 푎푛푛푥푛(푡), where the 푎푖푗’s are constants. This is a system of 푛 differential equations for the 푛 unknown functions 푥1(푡), . . . , 푥푛(푡). It’s important to note that the equations are ′ coupled, meaning the the expression for the derivative 푥푖(푡) contains not only 푥푖(푡), but (possibly) all the rest of the unknown functions. It’s unclear how to proceed using methods we’ve learned for scalar differential equation. Of course, to find a specific solution of (1.1), we need to specify initial conditions for the unknown functions at some value 푡0 of 푡,

푥1(푡0) = 푐1

푥2(푡0) = 푐2 (1.2) . .

푥푛(푡0) = 푐푛, where 푐1, 푐2, . . . , 푐푛 are constants. It’s pretty clear that is going to help here. We can put our unknown functions into a vector-valued function ⎡ ⎤ 푥1(푡) ⎢푥2(푡)⎥ 푥(푡) = ⎢ ⎥ , ⎢ . ⎥ ⎣ . ⎦ 푥푛(푡)

Version Time-stamp: ”2011-03-31 18:11:43 drager”. 1 2 LANCE D. DRAGER and our constant coefficients into a an 푛 × 푛 matrix 퐴 = [푎푖푗]. Recall that to differentiate a vector-valued functions, we differentiate each component, so ⎡ ′ ⎤ 푥1(푡) ′ ⎢푥2(푡)⎥ 푥′(푡) = ⎢ ⎥ , ⎢ . ⎥ ⎣ . ⎦ ′ 푥푛(푡) so we can rewrite (1.1) more compactly in vector-matrix form as (1.3) 푥′(푡) = 퐴푥(푡). If we put our initial values into a vector ⎡ ⎤ 푐1 ⎢푐2 ⎥ 푐 = ⎢ ⎥ , ⎢ . ⎥ ⎣ . ⎦ 푐푛 we can rewrite the initial conditions (1.2) as

(1.4) 푥(푡0) = 푐. Thus, the matrix form of our problem is

′ (1.5) 푥 (푡) = 퐴푥(푡), 푥(푡0) = 푐. A problem of this form is called an initial value problem (IVP). For information on the matrix manipulations used in these notes, see the Ap- pendix. Eigenvalues and eigenvectors are going to be important to our solution methods. Of course, even real matrices can have complex, nonreal eigenvalues. To take care of this problem, we’ll work with complex matrices and complex solutions to the differential equation from the start. In many (but not all) applications, one is only interested in real solutions, so we’ll indicate as we go along what happens when our matrix 퐴 and our initial conditions 푐 are real. The equation 푥′(푡) = 퐴푥(푡) is a homogeneous equation. An inhomogeneous equation would be one of the form 푥′(푡) = 퐴푥(푡) + 푓(푡), where 푓(푡) is a given vector-value function. We’ll discuss homogeneous systems to begin with, and show how to solve inhomogeneous systems at the end of these notes.

1.1. Notation. The symbol R will denote the real numbers and C will denote the complex numbers. We will denote by Mat푚×푛(C) the space of 푚 × 푛 matrices with entries in C and Mat푚×푛(R) will denote the space of 푚 × 푛 matrices with entries in R. 푛 We use R as a synonym Mat푛×1(R), the space of column vectors with 푛 entries. 푛 Similarly, C is a synonym for Mat푛×1(C). NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 3

2. The In this section, we’ll first consider the existence and uniqueness question for our system of equations. We’ll then consider the fundamental matrix for our system and show how this solves the problem. In the last subsection, we’ll view this fundamental matrix as a matrix exponential function.

2.1. Initial Value Problem and Existence and Uniqueness. The main prob- lem we’re interested in is the initial value problem ′ (2.1) 푥 (푡) = 퐴푥(푡), 푥(푡0) = 푐, 푛 were 퐴 ∈ Mat푛×푛(C), 푐 ∈ C and we’re solving for a function 푥(푡) with values in 푛 C , defined on some interval in R containing 푡0. We state the following existence and uniqueness theorem without proof. Theorem 2.1 (Existance and Uniqueness Theorem). Let 퐴 be an 푛 × 푛 complex 푛 matrix, let 푐 ∈ C and let 푡0 ∈ R. (1) There is a differentiable function 푥: R → C푛 : 푡 ↦→ 푥(푡) such that ′ 푥 (푡) = 퐴푥(푡), for all 푡 ∈ R,

푥(푡0) = 푐. 푛 (2) If 퐽 ⊆ R is an open interval in R that contains 푡0 and 푦 : 퐽 → C is a differentiable function such that 푦′(푡) = 퐴푦(푡), for all 푡 ∈ 퐽,

푦(푡0) = 푐, then 푦(푡) = 푥(푡) for all 푡 ∈ 퐽.

In view of this Theorem, we may as well consider solutions defined on all of R. For brevity, we’ll summarize by saying that solutions of the initial value problem are unique. It will turn out to be useful for consider initial value problems for matrix val- ued functions. To distinguish the cases, we’ll usually write 푋(푡) for our unknown function with values in Mat푛×푛(C).

Theorem 2.2. Suppose that 퐴 ∈ Mat푛×푛(C) and that 푡0 ∈ R. Let 퐶 be a fixed 푛 × 푛 complex matrix. Then there is a function 푋 : R → Mat푛×푛(C): 푡 ↦→ 푋(푡) such that ′ 푋 (푡) = 퐴푋(푡), 푡 ∈ R (2.2) 푋(푡0) = 퐶. This solution is unique in the sense of Theorem 2.1 Proof. If we write 푋(푡) in terms of its columns as

푋(푡) = [푥1(푡) | 푥2(푡) | · · · | 푥푛(푡)], so each 푥푖(푡) is a vector-valued function, then ′ ′ ′ ′ 푋 (푡) = [푥1(푡) | 푥2(푡) | · · · | 푥푛(푡)],

퐴푋(푡) = [퐴푥1(푡) | 퐴푥2(푡) | · · · | 퐴푥푛(푡)] 4 LANCE D. DRAGER

Thus, the matrix differential equation 푋′(푡) = 퐴푋(푡) is equivalent to 푛 vector differential equations ′ 푥1(푡) = 퐴푥1(푡) ′ 푥2(푡) = 퐴푥2(푡) . . ′ 푥푛(푡) = 퐴푥푛(푡). If we write the initial matrix 퐶 in terms of its columns as

퐶 = [푐1 | 푐2 | · · · | 푐푛], then the initial condition 푋(푡0) = 퐶 is equivalent to the vector equations

푥1(푡0) = 푐1

푥2(푡0) = 푐2 . .

푥푛(푡0) = 푐푛 Since each of the initial value problems ′ 푥푗(푡) = 퐴푥푗(푡), 푥푗(푡0) = 푐푗, 푗 = 1, 2, . . . , 푛 has a unique solution, we conclude that the matrix initial value problem (2.2) has a unique solution.  2.2. The Fundamental Matrix and It’s Properties. It turns out we only need to solve one matrix initial value problem in order to solve them all. Definition 2.3. Let 퐴 be a complex 푛 × 푛 matrix. The unique 푛 × 푛-matrix value function 푋(푡) that solves the matrix initial valued problem (2.3) 푋′(푡) = 퐴푋(푡), 푋(0) = 퐼 will be denoted by Φ퐴(푡), in order indicate the dependence on 퐴. In other words, Φ퐴(푡) is the unique function so that ′ (2.4) Φ퐴(푡) = 퐴Φ퐴(푡), Φ퐴(0) = 퐼.

The function Φ퐴(푡) is called the Fundamental Matrix of (2.3).

We’ll have a much more intuitive notation for Φ퐴(푡) in a bit, but we need to do some work first. First, let’s show that Φ퐴(푡) solves the initial value problems we’ve discussed so far. Theorem 2.4. Let 퐴 be an 푛 × 푛 complex matrix. (1) Let 푐 ∈ C푛. The solution of the initial value problem ′ (2.5) 푥 (푡) = 퐴푥(푡), 푥(푡0) = 푐, is

(2.6) 푥(푡) = Φ퐴(푡 − 푡0)푐. NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 5

(2) Let 퐶 ∈ Mat푛×푛(C). The solution 푋(푡) of the matrix initial value problem ′ (2.7) 푋 (푡) = 퐴푋(푡), 푋(푡0) = 퐶. is

(2.8) 푋(푡) = Φ퐴(푡 − 푡0)퐶. Proof. Consider the matrix-valued function

Ψ(푡) = Φ퐴(푡 − 푡0). We then have 푑 Ψ′(푡) = Φ (푡 − 푡 ) 푑푡 퐴 0 푑 = Φ′ (푡 − 푡 ) (푡 − 푡 ) 퐴 0 푑푡 0 = 퐴Φ(푡 − 푡0) = 퐴Ψ(푡).

We also have Ψ(푡0) = Φ퐴(푡0 − 푡0) = Φ퐴(0) = 퐼. For the first part of the proof, suppose 푐 is a constant vector, and let 푦(푡) = Ψ(푡)푐. Then 푦′(푡) = Ψ′(푡)푐 = 퐴Ψ(푡)푐 = 퐴푦(푡), and 푦(푡0) = Ψ(푡0)푐 = 퐼푐 = 푐. Thus, 푦(푡) is the unique solution of the initial value problem (2.5). The proof of the second part is very similar.  Exercise 2.5. Show that

Φ0(푡) = 퐼, for all 푡, where 0 is the 푛 × 푛 .

In the rest of this subsection, we’re going to derive some properties of Φ퐴(푡). The pattern of proof is the same in each case; we show two functions satisfy the same initial value problem, so they must be the same. Here’s a simple example to start.

Theorem 2.6. Let 퐴 be an 푛 × 푛 real matrix. Then Φ퐴(푡) is real. The solutions of the initial value problems (2.5) and (2.7) are real if the initial data, 푐 or 퐶, is real. Recall that the conjugate of a 푧 is denoted by푧 ¯. Of course, 푧 is real if and only if 푧 =푧 ¯. If 푥(푡) is a complex valued function, we define푥 ¯(푡) by 푥¯(푡) = 푥(푡). Note that 푑 푥¯′(푡) = 푥(푡) = 푥′(푡). 푑푡

Proof of Theorem. Let 퐴 be a complex matrix. The fundamental solution Φ퐴(푡) solves the IVP (2.9) 푋′(푡) = 퐴푋(푡), 푋(0) = 퐼 6 LANCE D. DRAGER

Suppose 푋(푡) is the solution of this IVP. Taking conjugates, we get (2.10) 푋¯ ′(푡) = 퐴¯푋¯(푡), 푋¯(0) = 퐼. In other words, (2.10) shows that 푋¯(푡) is the solution of the IVP (2.11) 푌 ′(푡) = 퐴푌¯ (푡), 푌 (0) = 퐼, ¯ so we must have 푌 (푡) = 푋(푡). But (2.11) is the IVP that defines Φ퐴¯(푡). In other words, for a complex matrix 퐴, we have

Φ퐴¯(푡) = Φ퐴(푡). Suppose now that 퐴 is real. Then 퐴¯ = 퐴. The last equation becomes

Φ퐴(푡) = Φ퐴(푡).

Since Φ퐴(푡) is equal to its conjugate, it must be real. Of course, if Φ퐴(푡) is real, and 푐 is a real vector, the solution 푥(푡) = Φ퐴(푡 − 푡0)푐 of (2.5) is real. A similar argument takes care of (2.7). 

The next properties is the basic property of Φ퐴(푡) that is often used. Theorem 2.7. Let 퐴 be an 푛 × 푛 complex matrix. For any 푡, 푠 ∈ R,

(2.12) Φ퐴(푡 + 푠) = Φ퐴(푡)Φ퐴(푠). Proof. Let 푠 be fixed by arbitrary, and think both sides of (2.12) as functions of 푡. Consider the matrix initial value problem ′ (2.13) 푋 (푡) = 퐴푋(푡), 푋(0) = Φ퐴(푠). According to the Theorem 2.4, the solution of this initial value problem is 푋(푡) = Φ퐴(푡)Φ퐴(푠). On the other hand, consider the function Ψ(푡) = Φ퐴(푡 + 푠). We have 푑 Ψ′(푡) = Φ (푡 + 푠) 푑푡 퐴 푑 = Φ′ (푡 + 푠) (푡 + 푠) 퐴 푑푡 = 퐴Φ퐴(푡 + 푠) = 퐴Ψ(푡) and Ψ(0) = Φ퐴(0 + 푠) = Φ퐴(푠). Thus, Ψ(푡) is also a solution to the initial value problem (2.13). Our two solutions must be the same, which proves the theorem.  Since 푡 + 푠 = 푠 + 푡, we readily obtain the following corollary. Corollary 2.8. For any real numbers 푡 and 푠,

Φ퐴(푡)Φ퐴(푠) = Φ퐴(푠)Φ퐴(푡), i.e., the matrices Φ퐴(푡) and Φ퐴(푠) commute. Almost as easily, we obtain a second corollary.

Corollary 2.9. For any 푡 ∈ R, the matrix Φ퐴(푡) is invertible and the inverse is Φ퐴(−푡). Proof.

Φ퐴(푡)Φ퐴(−푡) = Φ퐴(−푡)Φ퐴(푡) = Φ퐴(푡 + (−푡)) = Φ퐴(0) = 퐼.  NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 7

The next theorem has a similar proof. We’ll use this later. Theorem 2.10. Let 퐴 be an 푛 × 푛 complex matrix and let 푠 be a . Then Φ푠퐴(푡) = Φ퐴(푠푡), 푡 ∈ R.

Proof. Let Ψ(푡) = Φ퐴(푠푡). Then 푑 Ψ′(푡) = Φ (푠푡) 푑푡 퐴 푑 = Φ′ (푠푡) (푠푡) 퐴 푑푡 ′ = 푠Φ퐴(푠푡)

= 푠퐴Φ퐴(푠푡) = (푠퐴)Ψ(푡), and Ψ(0) = 퐼. Thus, Ψ(푡) satisfies the same initial value problem that characterizes Φ푠퐴(푡), so the two functions must be equal.  The next theorem describes how the effect of similarity transformations. This well be the key to computing Φ퐴(푡) when 퐴 is diagonalizable. Theorem 2.11. Let 퐴 be and 푛 × 푛 complex matrix, and let 푃 be an invertible 푛 × 푛 complex matrix. Then −1 Φ푃 −1퐴푃 (푡) = 푃 Φ퐴(푡)푃. −1 Proof. Let Ψ(푡) = 푃 Φ퐴(푡)푃 . Then we have ′ −1 ′ Ψ (푡) = 푃 Φ퐴(푡)푃 −1 = 푃 퐴Φ퐴(푡)푃 −1 −1 = 푃 퐴푃 푃 Φ퐴(푡)푃 = (푃 −1퐴푃 )Ψ(푡). We also have −1 −1 Ψ(0) = 푃 Φ퐴(0)푃 = 푃 퐼푃 = 퐼.

Thus, 휓(푡) is a solution of the same initial value problem that characterizes Φ푃 −1퐴푃 (푡), so the two functions are equal.  The following proposition is preparation for the next theorem. As we’ll see, what commutes with what is important in this subject. Proposition 2.12. Let 퐴 be an 푛 × 푛 complex matrix. Let 퐵 be an 푛 × 푛 complex matrix that commutes with 퐴, i.e., 퐴퐵 = 퐵퐴. Then 퐵 commutes with Φ퐴(푡), i.e.,

퐵Φ퐴(푡) = Φ퐴(푡)퐵, 푡 ∈ R. In particular, since 퐴 commutes with itself, we have

퐴Φ퐴(푡) = Φ퐴(푡)퐴, 푡 ∈ R. Proof. Consider the matrix initial value problem (2.14) 푋′(푡) = 퐴푋(푡), 푋(0) = 퐵. According to Theorem 2.4, the solution to this initial value problem is 푋(푡) = Φ퐴(푡)퐵. 8 LANCE D. DRAGER

Let Ψ(푡) = 퐵Φ퐴(푡). Then we have ′ ′ Ψ (푡) = 퐵Φ퐴(푡)

= 퐵퐴Φ퐴(푡)

= 퐴퐵Φ퐴(푡) since 퐴퐵 = 퐵퐴 = 퐴Ψ(푡).

We also have Ψ(0) = 퐵Φ퐴(0) = 퐵퐼 = 퐵. Thus, Ψ(푡) solves the initial value problem (2.14), so our two solutions must be equal.  Theorem 2.13. If 퐴 and 퐵 are 푛 × 푛 square matrices that commute, i.e., 퐴퐵 = 퐵퐴, then Φ퐴+퐵(푡) = Φ퐴(푡)Φ퐵(푡). Of course, since 퐴 + 퐵 = 퐵 + 퐴, we would have to have

Φ퐴+퐵(푡) = Φ퐵(푡)Φ퐴(푡), so the matrices Φ퐴(푡) and Φ퐵(푡) commute.

Proof. You know the drill by now. Let Ψ(푡) = Φ퐴(푡)Φ퐵(푡). Then we have ′ ′ ′ Ψ (푡) = Φ퐴(푡)Φ퐵(푡) + Φ퐴(푡)Φ퐵(푡) by the Product Rule

= 퐴Φ퐴(푡)Φ퐵(푡) + Φ퐴(푡)퐵Φ퐵(푡)

= 퐴Φ퐴(푡)Φ퐵(푡) + 퐵Φ퐴(푡)Φ퐵(푡) by Proposition 2.12

= (퐴 + 퐵)Φ퐴(푡)Φ퐵(푡) = (퐴 + 퐵)Ψ(푡).

In addition we have Ψ(0) = Φ퐴(0)Φ퐵(0) = 퐼퐼 = 퐼. Thus, Ψ(푡) solves the same initial value problem as Φ퐴+퐵(푡), so the two functions are equal.  Exercise 2.14. Under the hypotheses of the last theorem, show that

Φ퐴(푠)Φ퐵(푡) = Φ퐵(푡)Φ퐴(푠) for all 푠, 푡 ∈ R, not just 푠 = 푡 as stated in the theorem. 2.3. Matrix Exponentials. If 푎 is a number, consider the initial value problem 푥′(푡) = 푎푥, 푥(0) = 1. The solution is 푥(푡) = 푒푎푡. Of course this depends on knowing the exponential function exp(푥) = 푒푥. But this function can also be recovered from the solution of the IVP as 푒푎 = 푥(1). We can follow a similar course to define 푒퐴 = exp(퐴) when 퐴 is a matrix. Definition 2.15. If 퐴 is a complex 푛 × 푛 matrix, we define 퐴 푒 = Φ퐴(1). Now use Theorem 2.10 (reversing the roles of 푠 and 푡). For any 푠, we have

Φ푡퐴(푠) = Φ퐴(푡푠). Setting 푠 = 1 in this equation gives us 푡퐴 푒 = Φ푡퐴(1) = Φ퐴(푡). 푡퐴 For the rest of the notes, we’ll write 푒 or exp(푡퐴) instead of Φ퐴(푡). Let’s summarize the properties we’ve developed so far in the new notation. NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 9

Proposition 2.16. If 퐴 is a complex 푛×푛 matrix, the matrix exponential function 푒푡퐴 satisfies the following properties. (1) 푑 푒푡퐴 = 퐴푒푡퐴, 푒0퐴 = 퐼. 푑푡 (2) If 퐴 is real, 푒푡퐴 is real. (3) 푒(푡1+푡2)퐴 = 푒푡1퐴푒푡2퐴. (4) For each fixed 푡, 푒푡퐴 is invertible, and the inverse is 푒−푡퐴. (5) 푒푡(푠퐴) = 푒(푠푡)퐴 (6) For any invertible 푛 × 푛 matrix 푃 ,

−1 푃 −1푒푡퐴푃 = 푒푡푃 퐴푃 . (7) If 퐵 commutes with 퐴, then 퐵푒푡퐴 = 푒푡퐴퐵 (8) If 퐴 and 퐵 commute, then 푒푡(퐴+퐵) = 푒푡퐴푒푡퐵 = 푒푡퐵푒푡퐴. Remark 1. Of course, it would be nice to know what 푒푡(퐴+퐵) is when 퐴 and 퐵 don’t commute. The answer is given by the Baker-Campbell-Hausdorff formula, which is fairly complex. This is beyond the scope of these notes. There is another approach to developing the matrix exponential function, which we should mention. Recall that, for a real variable 푥, the exponential function 푒푥 is given by a power series, namely ∞ ∑︁ 1 1 1 1 푒푥 = 푥푘 = 1 + 푥 + 푥2 + 푥3 + 푥4 + ··· . 푘! 2 6 24 푘=0 If 퐴 is a , then formally substituting 푥 = 푡퐴 in the series above would suggest that we define 푒푡퐴 by ∞ ∑︁ 푡푘 1 1 (2.15) 푒푡퐴 = 퐴푘 = 퐼 + 푡퐴 + 푡2퐴2 + 푡3퐴3 + ··· . 푘! 2 6 푘=0 Of course, this is an infinite sum of matrices. One can interpret convergence of this series to mean that the scalar infinite series one gets in each slot converges. Of course, to make this proposed definition valid, we would have to show that the series (2.15) converges. To make the definition useful, we would have to develop enough properties of such infinite series to prove that 푒푡퐴 has the properties in Proposition 2.16. We’ve decided to take a different approach. Still, we’ll see an echo of the power series (2.15) below.

3. Computing Matrix Exponentials We will first consider how to compute 푒푡퐴 when 퐴 is diagonalizable, and the consider the nondiagonalizable case. 10 LANCE D. DRAGER

3.1. The Diagonalizable Case. Recall that an 푛 × 푛 matrix 퐴 is diagonalizable if there is an 푃 so that 푃 −1퐴푃 is a matrix 퐷. This means that 퐷 is of the form ⎡ ⎤ 휆1 ⎢ 휆2 ⎥ 퐷 = ⎢ ⎥ , ⎢ .. ⎥ ⎣ . ⎦ 휆푛 where all the off diagonal entries are zero. The first thing to do is to find 푒푡퐷 when 퐷 is diagonal. Proposition 3.1. If 퐷 is the ⎡ ⎤ 휆1 ⎢ 휆2 ⎥ 퐷 = ⎢ ⎥ , ⎢ .. ⎥ ⎣ . ⎦ 휆푛 then 푒푡퐷 is the diagonal matrix ⎡푒휆1푡 ⎤ ⎢ 푒휆2푡 ⎥ 푒푡퐷 = ⎢ ⎥ . ⎢ .. ⎥ ⎣ . ⎦ 푒휆푛푡 Proof. We just check that our proposed solution satisfied the right initial value problem. So, define Ψ(푡) by ⎡푒휆1푡 ⎤ ⎢ 푒휆2푡 ⎥ Ψ(푡) = ⎢ ⎥ . ⎢ .. ⎥ ⎣ . ⎦ 푒휆푛푡 Then, ⎡푒휆10 ⎤ ⎡1 ⎤ ⎢ 푒휆20 ⎥ ⎢ 1 ⎥ Ψ(0) = ⎢ ⎥ = ⎢ ⎥ = 퐼 ⎢ .. ⎥ ⎢ .. ⎥ ⎣ . ⎦ ⎣ . ⎦ 푒휆푛0 1 By differentiating each component, we get

⎡ 휆1푡 ⎤ 휆1푒 휆1푡 ⎢ 휆2푒 ⎥ Ψ′(푡) = ⎢ ⎥ . ⎢ .. ⎥ ⎣ . ⎦ 휆푛푡 휆푛푒 One the other hand

⎡ ⎤ ⎡ 휆1푡 ⎤ ⎡ 휆1푡 ⎤ 휆1 푒 휆1푒 휆2푡 휆1푡 ⎢ 휆2 ⎥ ⎢ 푒 ⎥ ⎢ 휆2푒 ⎥ 퐷Ψ(푡) = ⎢ ⎥ ⎢ ⎥ = ⎢ ⎥ . ⎢ .. ⎥ ⎢ .. ⎥ ⎢ .. ⎥ ⎣ . ⎦ ⎣ . ⎦ ⎣ . ⎦ 휆푛푡 휆푛푡 휆푛 푒 휆푛푒 NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 11

Thus, Ψ′(푡) = 퐷Ψ(푡). We conclude that Ψ(푡) solves the initial value problem that defines 푒푡퐷, so we 푡퐷 must have 푒 = Ψ(푡).  We can now easily compute 푒푡퐴 if 퐴 is diagonalizable. In this case, we have 푃 −1퐴푃 = 퐷 for some diagonal matrix 퐷. We can also write this equation as 퐴 = 푃 퐷푃 −1. But then, we know −1 푒푡퐴 = 푒푡푃 퐷푃 = 푃 푒푡퐷푃 −1. So, to compute 푒푡퐴 when 퐴 is diagonalizable, we use the equation (3.1) 푒푡퐴 = 푃 푒푡퐷푃 −1. Let’s do a couple of computations. Example 3.2. Consider the matrix ⎡ 8 −4 −3 ⎤ 퐴 := ⎣ 6 −2 −3 ⎦ 6 −4 −1 The characteristic polynomial of 퐴 is 푝(휆) = 휆3 − 5휆2 + 8휆 − 4 which factors as 푝(휆) = (휆 − 1)(휆 − 2)2. Thus the eigenvalues are 1 and 2. First, let’s find a basis for 퐸(1), the eigenspace for eigenvalue 1. Of course, 퐸(1) is the nullspace of 퐴 − (1)퐼. We calculate that ⎡ 7 −4 −3 ⎤ 퐴 − 퐼 = ⎣ 6 −3 −3 ⎦ 6 −4 −2 The reduced of 퐴 − 퐼 is ⎡ 1 0 −1 ⎤ 푅 = ⎣ 0 1 −1 ⎦ 0 0 0 By the usual method, we find the nullspace of 푅, which is the same as the nullspace of 퐴 − 퐼. The conclusion is that 퐸(1) is one dimensional with basis ⎡ 1 ⎤ 푣1 = ⎣ 1 ⎦ . 1 Next, let’s look for a basis of 퐸(2). We calculate ⎡ 6 −4 −3 ⎤ 퐴 − 2퐼 = ⎣ 6 −4 −3 ⎦ . 6 −4 −3 The reduced row echelon form of 퐴 − 2퐼 is ⎡ 1 −2/3 −1/2 ⎤ 푅 = ⎣ 0 0 0 ⎦ . 0 0 0 12 LANCE D. DRAGER

By the usual method, we find that 퐸(2) is two dimensional with basis ⎡ 1/2 ⎤ ⎡ 2/3 ⎤ 푢2 = ⎣ 0 ⎦ , 푢3 = ⎣ 1 ⎦ . 1 0 It’s not necessary, but we can make things look nicer by getting rid of the fractions. It we multiply 푢2 by 2 and 푢3 by 3, we’ll still have a basis. This gives us a new basis ⎡ 1 ⎤ ⎡ 2 ⎤ 푣2 = 2푢2 = ⎣ 0 ⎦ , 푣3 = 3푢3 = ⎣ 3 ⎦ 2 0 Since we have 3 independent eigenvectors, the matrix 퐴 is diagonalizable. We put the basis vectors into a matrix as columns, so we get ⎡ 1 1 2 ⎤ 푃 = [푣1 | 푣2 | 푣3] = ⎣ 1 0 3 ⎦ 1 2 0 The corresponding diagonal matrix is ⎡1 0 0⎤ 퐷 = ⎣0 2 0⎦ . 0 0 2 The reader is invited to check that 푃 −1퐴푃 = 퐷. More importantly at the moment, we have 퐴 = 푃 퐷푃 −1. Thus, 푒푡퐴 = 푃 푒푡퐷푃 −1. We easily calculate1 푒푡퐷 as ⎡ exp (푡) 0 0 ⎤ ⎣ 0 exp (2푡) 0 ⎦ 0 0 exp (2푡) Using a machine, we compute 푒푡퐴 = 푃 푒푡퐷푃 −1 ⎡ 1 1 2 ⎤ ⎡ exp (푡) 0 0 ⎤ ⎡ −6 4 3 ⎤ = ⎣ 1 0 3 ⎦ ⎣ 0 exp (2푡) 0 ⎦ ⎣ 3 −2 −1 ⎦ 1 2 0 0 0 exp (2푡) 2 −1 −1 ⎡ −6 exp (푡) + 7 exp (2푡) 4 exp (푡) − 4 exp (2푡) 3 exp (푡) − 3 exp (2푡) ⎤ = ⎣ −6 exp (푡) + 6 exp (2푡) 4 exp (푡) − 3 exp (2푡) 3 exp (푡) − 3 exp (2푡) ⎦ . −6 exp (푡) + 6 exp (2푡) 4 exp (푡) − 4 exp (2푡) 3 exp (푡) − 2 exp (2푡) Consider the initial value problem ⎡ 1 ⎤ ′ 푥 (푡) = 퐴푥(푡), 푥(0) = 푐 = ⎣ 2 ⎦ . −3 The solution is ⎡ −7 exp (푡) + 8 exp (2푡) ⎤ 푡퐴 푥(푡) = 푒 푐 = ⎣ −7 exp (푡) + 9 exp (2푡) ⎦ . −7 exp (푡) + 4 exp (2푡)

1The software I’m using prefers exp(푡) to 푒푡 NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 13

Example 3.3. Here is a simple example with complex eigenvalues. Let [︃ 4 −1 ]︃ 퐴 = . 5 0 The characteristic polynomial is 푝(휆) = det(퐴 − 휆퐼) = 휆2 − 4휆 + 5. The roots of the characteristic polynomial are 2 + 푖 and 2 − 푖. We calculate [︃ 2 − 푖 −1 ]︃ 퐴 − (2 + 푖)퐼 = . 5 −2 − 푖 The reduced row echelon form of 퐴 − (2 + 푖)퐼 is [︃ 1 −2/5 − (1/5) 푖 ]︃ 푅 = . 0 0 By the usual method we find the nullspace of 푅, which is the same as the nullspace of 퐴 − (2 + 푖)퐼. The result is that the nullspace is one dimensional with basis [︃ 2/5 + (1/5) 푖 ]︃ 푣1 = . 1

Thus, 푣1 is a basis of the eigenspace of 퐴 for eigenvalue 2+ 푖. The other eigenvalue, 2 − 푖 is the conjugate of the first eigenvalue, so we can just conjugate the basis we found for 2 + 푖. This give us [︃ 2/5 − (1/5) 푖 ]︃ 푣2 = 1 and a basis for the eigenspace for eigenvalue 2 − 푖. Since we have two independent eigenvectors, the matrix 퐴 is diagonalizable. If we put 푣1 and 푣2 into a matrix we get [︃ 2/5 + (1/5) 푖 2/5 − (1/5) 푖 ]︃ 푃 = . 1 1 The corresponding diagonal matrix is [︃ 2 + 푖 0 ]︃ 퐷 = . 0 2 − 푖 Note that each column of 푃 is an eigenvector for the eigenvalue occurring in the corresponding column of 퐷. You can now check that 푃 −1퐴푃 = 퐷, or 퐴 = 푃 퐷푃 −1. Since 퐴 is real, we know that 푒푡퐴 must be real, even though 푃 and 퐷 are not real. We compute 푒푡퐷 as [︂푒(2+푖)푡 0 ]︂ 푒푡퐷 = 0 푒(2−푖)푡 We then have 푒푡퐴 = 푃 푒푡퐷푃 −1. Putting into the TI-89 gives [︂푒2푡(cos(푡) + 2 sin(푡)) 푒2푡 sin(푡) ]︂ (3.2) 푒푡퐴 = . 5푒2푡 sin(푡) 푒2푡(cos(푡) − 2 sin(푡)) 14 LANCE D. DRAGER

Of course, the calculator has used Euler’s Formula 푒(훼+푖훽)푡 = 푒훼푡 cos(훽푡) + 푖푒훼푡 sin(훽푡). Sometimes you have to put this in by hand to persuade recalcitrant software to give you results that are clearly real. If we are given an initial condition [︂푐 ]︂ 푥(0) = 푐 = 1 , 푐2 the solution of the initial value problem 푥′(푡) = 퐴푥(푡), 푥(0) = 푐 is [︃ (︀ 2 푡 2 푡 )︀ 2 푡 ]︃ 푒 cos (푡) + 2 푒 sin (푡) 푐1 − 푒 sin (푡) 푐2 푥(푡) = 푒푡퐴푐 = . 2 푡 (︀ 2 푡 2 푡 )︀ 5 푒 sin (푡) 푐1 + 푒 cos (푡) − 2 푒 sin (푡) 푐2 3.2. Matrices. In this subsection we discuss nilpotent matrices, which will be used in the next subsection to compute the matrix exponential of a general nondiagonalizable matrix. Definition 3.4. A square matrix 푁 is said to be nilpotent if 푁 푝 = 0 for some 푝 ≥ 1. Example 3.5. Consider the matrix [︂0 1]︂ 퐽 = . 2 0 0 2 Of course, 퐽2 ̸= 0, but the reader can easily check that 퐽2 = 0. If we go one dimension higher, we have ⎡0 1 0⎤ 퐽3 = ⎣0 0 1⎦ . 0 0 0 2 3 The reader can easily check that 퐽3 ̸= 0 but 퐽3 = 0. You can see a pattern here, but these are just some simple examples of nilpotent matrices. We’ll see later that it’s possible to have a nilpotent matrix where all the entries are nonzero. Of course, if 푁 푝 = 0, then 푁 푞 = 0 for all 푞 > 푝. One might wonder how high a power 푝 is necessary to make 푁 푝 = 0. This is answered in the next theorem. Theorem 3.6. Let 푁 be a nilpotent 푛 × 푛 matrix. Then 푁 푛 = 0. To put it more precisely, if 푁 푝−1 ̸= 0 and 푁 푝 = 0, then 푝 ≤ 푛. The proof of this theorem will be given in the appendices. Our main concern here is to compute 푒푡푁 when 푁 is nilpotent. This can be done by a simple matrix calculation. Theorem 3.7. Let 푁 be an 푛 × 푛 matrix. Then 푛−1 1 1 1 ∑︁ 1 (3.3) 푒푡푁 = 퐼 + 푡푁 + 푡2푁 2 + 푡3푁 3 + ··· + 푡푛−1푁 푛−1 = 푡푘푁 푘. 2! 3! (푛 − 1)! 푘! 푘=0 NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 15

Note that it may well happen that 푁 푝 = 0 for some 푝 < 푛 − 1, so some of the terms above will vanish. You may recognize this formula as what you would get if you plug 푁 into the power series (2.15) and notice that all the terms with exponent higher than 푛 − 1 will be zero. However, we can prove the theorem without resorting to the power series, we just use our usual method.

Proof of Theorem. Consider the function

1 1 1 1 Ψ(푡) = 퐼 + 푡푁 + 푡2푁 2 + 푡3푁 3 + ··· + 푡푛−2푁 푛−2 + 푡푛−1푁 푛−1. 2! 3! (푛 − 2)! (푛 − 1)!

Clearly Ψ(0) = 퐼. It’s easy to differentiate Ψ(푡) if you recall that 푝/푝! = 1/(푝 − 1)!. The result is

1 1 1 (3.4) Ψ′(푡) = 푁 + 푡푁 2 + 푡2푁 3 + ··· + 푡푛−3푁 푛−2 + 푡푛−2푁 푛−1. 2 (푛 − 3)! (푛 − 2)!

One the other hand, we can compute 푁Ψ(푡) by multiplying 푁 through the formula for Ψ(푡). The result is (3.5) 1 1 1 1 푁Ψ(푡) = 푁 +푡푁 2 + 푡2푁 3 + 푡3푁 4 +···+ 푡푛−2푁 푛−1 + +푡푛−1푁 푛 2! 3! (푛 − 2)! (푛 − 1)!

But the last term is zero, because 푁 푛 = 0. Thus, we get exactly the same formula in (3.4) and (3.5), so we have Ψ′(푡) = 푁Ψ(푡). Thus, Ψ(푡) satisfies the same initial 푡푁 푡푁 value problem that defines 푒 . Thus, we must have 푒 = Ψ(푡). 

With this formula, we can compute 푒푡푁 for (reasonably small) nilpotent matrices.

Example 3.8. Consider the matrix

⎡0 1 0 0⎤ ⎢0 0 1 0⎥ 퐽 = ⎢ ⎥ . ⎣0 0 0 1⎦ 0 0 0 0

We have

⎡0 0 1 0⎤ 2 ⎢0 0 0 1⎥ 퐽 = ⎢ ⎥ ⎣0 0 0 0⎦ 0 0 0 0 ⎡0 0 0 1⎤ 3 ⎢0 0 0 0⎥ 퐽 = ⎢ ⎥ ⎣0 0 0 0⎦ 0 0 0 0 퐽 4 = 0. 16 LANCE D. DRAGER

Thus, we can calculate 1 1 푒푡퐽 = 퐼 + 푡퐽 + 푡2퐽 2 + 푡3퐽 3 2 3! ⎡1 0 0 0⎤ ⎡0 1 0 0⎤ ⎡0 0 1 0⎤ ⎡0 0 0 1⎤ ⎢0 1 0 0⎥ ⎢0 0 1 0⎥ 1 2 ⎢0 0 0 1⎥ 1 3 ⎢0 0 0 0⎥ = ⎢ ⎥ + 푡 ⎢ ⎥ + 푡 ⎢ ⎥ + 푡 ⎢ ⎥ ⎣0 0 1 0⎦ ⎣0 0 0 1⎦ 2 ⎣0 0 0 0⎦ 6 ⎣0 0 0 0⎦ 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 ⎡1 푡 푡2/2 푡3/6⎤ ⎢0 1 푡 푡2/2⎥ = ⎢ ⎥ . ⎣0 0 1 푡 ⎦ 0 0 0 1 Example 3.9. Consider the matrix [︂3 −9]︂ 푁 = . 1 −3 The reader is invited to check that 푁 2 = 0. Thus, 푒푡푁 = 퐼 + 푡푁 [︂1 0]︂ [︂3 −9]︂ = + 푡 0 1 1 −3 [︂1 + 3푡 −9푡 ]︂ = . 푡 1 − 3푡

In general, it’s clear from the formula (3.3) that the entries of 푒푡푁 are polynomials in 푡.

3.3. Computing Exponentials with the Jordan Decomposition. In this sec- tion will see a “general” method for computing the matrix exponential of any ma- trix. I put “general” in quotes because we’re making our usual assumption that you can find the eigenvalues. Which matrices commute with each other is going to be important, so we start with an easy proposition about commuting matrices. The proof is left to the reader Theorem 3.10. Consider 푛×푛 matrices. We say 퐴 commutes with 퐵 if 퐴퐵 = 퐵퐴. (1) 퐴 commutes with itself and with the identity 퐼. (2) If 퐴 commutes with 퐵, 퐴 commutes with any power of 퐵. (3) If 퐴 commutes with 퐵 and with 퐶 then 퐴 commutes with 훼퐵 + 훽퐶 for any scalars 훼, 훽 ∈ C. If a matrix does not have enough independent eigenvectors it’s not diagonaliz- able. We need to look for more vectors to add to the list. It turns out the the important thing to look for is generalized eigenvectors. Definition 3.11. Let 퐴 be an 푛 × 푛 matrix and let 휆 be an eigenvalue of 퐴. The generalized eigenspace of 퐴 for eigenvalue 휆 is denoted by 퐺(휆) and is defined as 퐺(휆) = nullspace(︀(퐴 − 휆퐼)푛)︀. NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 17

Note that if (퐴 − 휆퐼)푝푣 = 0 then (퐴 − 휆퐼)푞푣 = 0 for all 푞 > 푝. In particular, the eigenspace of 퐴 for eigenvalue 휆 is defined by 퐸(휆) = nullspace((퐴 − 휆퐼)), so 퐸(휆) ⊆ 퐺(휆). In particular 퐺(휆) ̸= {0}. The following Big Theorem will justify our constructions in this section. It is stated without proof. Theorem 3.12 (Generalized Eigenspaces Theorem). Let 퐴 be an 푛×푛 matrix and let 휆1, . . . , 휆푘 be the distinct eigenvalues of 퐴. If you find a basis for each of the generalized eigenspaces 퐺(휆푗) and concatenate this lists into one long list of vectors, you get a basis for C푛. To make this notationally specific suppose that you take a basis 푣푗, 푣푗, . . . , 푣푗 of 1 2 푛푗 퐺(휆푗), where 푛푗 is the dimension of 퐺(휆푗). The the list of vectors (3.6) 푣1, 푣1 . . . , 푣1 , 푣2, 푣2, . . . 푣2 , . . . , 푣푘, 푣푘, . . . , 푣푘 1 2 푛1 1 2 푛2 1 2 푛푘 is a basis of C푛. In particular, the dimensions of the generalized eigenspaces add up to to 푛. Corollary 3.13 (Generalized Eigenspaces Decomposition). Let 퐴 be an 푛 × 푛 푛 matrix and let 휆1, . . . , 휆푘 be the distinct eigenvalues of 퐴. Then every vector 푣 ∈ C can be written uniquely as

(3.7) 푣 = 푣1 + 푣2 + ··· + 푣푘, 푣푗 ∈ 퐺(휆푗). In particular, if

(3.8) 0 = 푣1 + 푣2 + ··· + 푣푘, 푣푗 ∈ 퐺(휆푗) then each 푣푗 is zero.

Proof. According to our big theorem, we can find a basis of C푛 of the form (3.6). If 푣 is any vector it can be written as a linear combination of the basis vectors. In particular, we can write

푘 푛푗 ∑︁ ∑︁ 푗 푗 (3.9) 푣 = 푐푝푣푝, 푗= 푝=1

푗 for some scalars 푐푝. But the term

푛푗 ∑︁ 푗 푗 푐푝푣푝 푝=1

푗 is in 퐺(휆푗), so (3.9) gives us the expression (3.7). Since the coefficients 푐푝 are unique, so is the decomposition (3.7). The second statement follows by uniqueness.  In preparation for our construction of the Jordan Decomposition, we have the following fact. Proposition 3.14. Let 퐴 be an 푛 × 푛 matrix. Then generalized eigenspaces of 퐴 are invariant under 퐴. In other words if 휆 is an eigenvalue of 퐴, 푣 ∈ 퐺(휆) =⇒ 퐴푣 ∈ 퐺(휆). 18 LANCE D. DRAGER

Proof. Suppose that 푣 ∈ 퐺(휆). Then, by definition, (퐴 − 휆퐼)푛푣 = 0 To check if 퐴푣 is in 퐺(휆), we have to check if (퐴 − 휆퐼)푛(퐴푣) is zero. But 퐴 and (퐴 − 휆퐼)푛 commute. So we have (퐴 − 휆퐼)푛(퐴푣) = ((퐴 − 휆퐼)푛퐴)푣 = (퐴(퐴 − 휆퐼)푛)푣 = 퐴((퐴 − 휆퐼)푛푣) = 퐴0 = 0.

Thus, 퐴푣 ∈ 퐺(휆).  We’re now ready to discuss the Jordan Decomposition Theorem and how to use it to compute matrix exponentials. Theorem 3.15 (Jordan Decomposition Theorem). Let 퐴 be an 푛×푛 matrix. Then there are unique matrices 푆 and 푁 that satisfy the following conditions (1) 퐴 = 푆 + 푁 (2) 푆푁 = 푁푆, i.e., 푁 and 푆 commute (it follows from the previous condition that 푆 and 푁 commute with 퐴). (3) 푆 is diagonalizable (4) 푁 is nilpotent. The expression 퐴 = 푆 + 푁 is the Jordan Decomposition of 퐴. We’ll leave the uniqueness part of this Theorem to the appendices. We’ll show how to construct 푆 and 푁, which will prove the existence part of the Theorem. This Theorem will enable us to compute 푒푡퐴. Since 푆 and 푁 commute, 푒푡퐴 = 푒푡푆푒푡푁 = 푒푡푁 푒푡푆. We know how to compute 푒푡푆, since 푆 is diagonalizable, and we know how to compute 푒푡푁 , since 푁 is nilpotent. We’ll give one application of the uniqueness part of the Theorem. Corollary 3.16. If 퐴 is a real matrix with Jordan Decomposition 퐴 = 푆 +푁, then 푆 and 푁 are real. Proof. In general, if 퐴 = 푆 + 푁, then 퐴¯ = 푆¯ + 푁¯. Taking the conjugates of both sides in 푆푁 = 푁푆 gives 푆¯푁¯ = 푁¯푆¯. Taking the conjugates of both sides in 푁 푛 = 0 shows 푁¯ 푛 = 0, so 푁¯ is nilpotent. We claim that 푆¯ is diagonalizable. First observe that if 푃 is invertible, we have 푃 푃 −1 = 푃 −1푃 = 퐼. Taking conjugates in this shows that 푃¯푃 −1 = 푃 −1푃¯ = 퐼¯ = 퐼. This shows that 푃¯−1 = 푃 −1. Now,since 푆 is invertible, there is an invertible matrix 푃 and a diagonal matrix 퐷 so that 푃 −1푆푃 = 퐷. Taking conjugates on both sides of this equation and using our previous remark, we have 푃¯−1푆¯푃¯ = 퐷.¯ The matrix 퐷¯ is certainly diagonal, so 푆¯ is diagonalizable. NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 19

Thus, 푆¯ and 푁¯ satisfy all the required conditions to form a Jordan Decomposition of 퐴¯. Thus, in general, 퐴¯ = 푆¯ + 푁¯ is the Jordan Decomposition of 퐴¯. But if 퐴 is real, then 퐴¯ = 퐴, so 퐴 = 푆¯ + 푁¯ where 푆¯ and 푁¯ satisfy all the right conditions. By the uniqueness part of the Jordan Decomposition Theorem, we must have 푆 = 푆¯ and 푁 = 푁¯, so both 푆 and 푁 are real.  We’ll now see how to construct the Jordan Decomposition, and then how to compute 푒푡퐴. To construct the Jordan Decomposition of 퐴, we first find the eigenvalues of 퐴, call them 휆1, . . . , 휆푘. Next, we find a basis of each of the generalized eigenspaces 퐺(휆푗). We know how to do this because 푛 퐺(휆푗) = nullspace((퐴 − 휆퐼) ). So, we compute the matrix (퐴 − 휆퐼)푛 and use our usual algorithm to find the null space. So, we first find the RREF of (퐴 − 휆퐼)푛 and we can then read off a basis of the nullspace. Obviously this will all be much pleasanter if we have machine assistance, like the TI-89 or Maple. Once we have found a basis of each generalized eigenspace, putting all the lists together gives a basis of C푛. Thus, if we insert all of these vector as the columns of a matrix 푃 , the matrix 푃 in 푛 × 푛 and is invertible. Now construct a diagonal matrix ⎡ ⎤ 휇1 ⎢ 휇2 ⎥ (3.10) 퐷 = ⎢ ⎥ , (off diagonal entries are 0), ⎢ .. ⎥ ⎣ . ⎦ 휇푛 where 휇ℓ is the eigenvalue that goes with column ℓ of 푃 , i.e., 휇ℓ = 휆푗 if column ℓ of 푃 is one of the basis vectors of 퐺(휆푗). So, all of the 휇ℓ’s are eigenvalues of 퐴, but the same eigenvalue could be repeated in several columns. We then construct a matrix 푆 by (3.11) 푆 = 푃 퐷푃 −1. Certainly 푆 is diagonalizable!

Claim. If 푣 ∈ 퐺(휆ℓ), then 푆푣 = 휆ℓ푣. Proof of Claim. This is just the way we constructed it. Describe 푃 by its columns as [︀ ]︀ 푃 = 푝1 푝2 ... 푝푛 . Since 푆푃 = 푃 퐷, we have 푆푝푗 = 휇푗푝푗.

By the construction, a subset of the columns of 푃 form a basis of 퐺(휆ℓ). Thus, we can find some columns

푝푗1 , 푝푗2 , . . . , 푝푗푚 , 20 LANCE D. DRAGER that form a basis, where 푚 is the dimension of 퐺(휆ℓ). Since these vectors are 퐺(휆ℓ), we have

휇푗1 = 휇푗2 = ··· = 휇푗푚 = 휆ℓ.

If 푣 ∈ 퐺(휆ℓ), we can write 푣 in terms of our basis as

푣 = 푐1푝푗1 + 푐2푝푗2 + ··· + 푐푚푝푗푚 . for some scalars 푐1, 푐2, . . . , 푐푚. Then we have

푆푣 = 푐1푆푝푗1 + 푐2푆푝푗2 + ··· + 푐푚푆푝푗푚

= 푐1휆ℓ푝푗1 + 푐2휆ℓ푝푗2 + ··· + 푐푚휆ℓ푝푗푚

= 휆ℓ(푐1푝푗1 + 푐2푝푗2 + ··· + 푐푚푝푗푚 )

= 휆푗푣.

This completes the proof of the claim.  If 푣 is any vector, we can write it in the form

(3.12) 푣 = 푣1 + ··· + 푣푘, 푣푗 ∈ 퐺(휆푗). Applying 푆 gives us

(3.13) 푆푣 = 휆1푣1 + 휆2푣2 + ··· + 휆푘푣푘, 휆푗푣푗 ∈ 퐺(휆푗). Claim. 푆 commutes with 퐴.

Proof of Claim. Recall that if 푣 ∈ 퐺(휆ℓ), then 퐴푣 ∈ 퐺(휆ℓ). If 푣 is any vector, we can write it in the form (3.12). Multiplying by 퐴 gives us

퐴푣 = 퐴푣1 + 퐴푣2 + ··· + 퐴푣푘.

Since each 퐴푣푗 is in 퐺(휆푗), we can apply 푆 to get

푆퐴푣 = 휆1퐴푣1 + 휆2퐴푣2 + ··· + 휆푘퐴푣푘. On the other hand,

푆푣 = 휆1푣1 + 휆2푣2 + ··· + 휆푘푣푘, and multiplying by 퐴 gives

퐴푆푣 = 퐴(휆1푣1 + 휆2푣2 + ··· + 휆푘푣푘) = 휆1퐴푣1 + 휆2퐴푣2 + ··· + 휆푘퐴푣푘

Thus, 푆퐴 = 퐴푆.  Now, we define 푁 by 푁 = 퐴 − 푆. Obviously 퐴 = 푆 + 푁. Since 푆 commutes with 퐴 and itself, 푆 commutes with 퐴 − 푆 = 푁. Thus, 푆푁 = 푁푆. It remains to show that 푁 is nilpotent.

Claim. If 푣 ∈ 퐺(휆ℓ), then 푛 푛 (3.14) (퐴 − 푆) 푣 = (퐴 − 휆ℓ) 푣 = 0. Proof of Claim. Since 퐴 and 푆 commute with each other and with the identity, 퐴 − 푆 and 퐴 − 휆ℓ퐼 commute. If 푣 ∈ 퐺(휆ℓ), 푆푣 = 휆ℓ푣. Thus,

(퐴 − 푆)푣 = 퐴푣 − 푆푣 = 퐴푣 − 휆ℓ푣 = (퐴 − 휆ℓ퐼)푣. NOTES ON SOLVING LINEAR SYSTEMS OF DIFFERENTIAL EQUATIONS 21

For the next power we have (퐴 − 푆)2푣 = (퐴 − 푆)[(퐴 − 푆)푣]

= (퐴 − 푆)[(퐴 − 휆ℓ퐼)푣]

= {(퐴 − 푆)(퐴 − 휆ℓ푣)} 푣

= {(퐴 − 휆ℓ퐼)(퐴 − 푆)} 푣

= (퐴 − 휆ℓ퐼)[(퐴 − 푆)푣]

= (퐴 − 휆ℓ퐼)[(퐴 − 휆ℓ퐼)푣] 2 = (퐴 − 휆ℓ퐼) 푣 푝 푝 We can continue inductively to show (퐴 − 푆) 푣 = (퐴 − 휆ℓ) 푣 for any power 푝. If we set 푝 = 푛, we get(3.14).  We can now prove that 푁 is nilpotent. If 푣 is any vector, we write it in the form

(3.15) 푣 = 푣1 + ··· + 푣푘, 푣푗 ∈ 퐺(휆푗). Applying 푁 푛 to both sides of (3.15) gives 푁 푛푣 = (퐴 − 푆)푛푣 푛 푛 푛 = (퐴 − 푆) 푣1 + (퐴 − 푆) 푣2 + ··· + (퐴 − 푆) 푣푘 푛 푛 푛 = (퐴 − 휆1) 푣1 + (퐴 − 휆2) 푣2 + ··· + (퐴 − 휆푘) 푣푘 = 0 + 0 + ··· + 0 = 0. Thus, 푁 푛푣 = 0 for all 푣, so 푁 푛 = 0. We’ve now constructed matrices 푆 and 푁 so that 퐴 = 푆 + 푁, 푆푁 = 푁푆, 푆 is diagonalizable and 푁 is nilpotent. So, we’ve found the Jordan Decomposition of 퐴. We can use the machinery we developed to compute 푒푡퐴. Our matrix 퐷 in (3.10) is diagonal, so ⎡푒휇1푡 ⎤ ⎢ 푒휇2푡 ⎥ 푒푡퐷 = ⎢ ⎥ , (off diagonal entries are 0), ⎢ .. ⎥ ⎣ . ⎦ 푒휇푛푡 Since 푆 = 푃 퐷푃 −1, we can compute 푒푡푆 by 푒푡푆 = 푃 푒푡퐷푃 −1. Since 푁 is nilpotent, we know how to compute 푒푡푁 . We can then compute 푒푡퐴 as 푒푡퐴 = 푒푡푆푒푡푁 = 푒푡푁 푒푡푆. Let’s summarize the steps in this process. Theorem 3.17 (Computing 푒푡퐴 ). Let 퐴 be an 푛 × 푛 matrix. To compute 푒푡퐴, follow these steps. (1) Find the characteristic polynomial of 퐴 and the eigenvalues of 퐴. Call the eigenvalues 휆1, . . . , 휆푘. (2) For each eigenvalue 휆ℓ, find a basis of 퐺(휆ℓ) by computing the matrix 퐵 = 푛 (퐴 − 휆ℓ) and the finding a basis for the nullspace of 퐵. (3) Put the basis vectors you’ve constructed in as the columns of a matrix 푃 . 22 LANCE D. DRAGER

(4) Construct the diagonal matrix ⎡ ⎤ 휇1 ⎢ 휇2 ⎥ 퐷 = ⎢ ⎥ , (off diagonal entries are 0), ⎢ .. ⎥ ⎣ . ⎦ 휇푛

where 휇푗 is the eigenvalue such that the 푗th column of 푃 is in 퐺(휇푗). (5) Construct the matrices 푆 = 푃 퐷푃 −1 and 푁 = 퐴 − 푆. Then 퐴 = 푆 + 푁 is the Jordan Decomposition of 퐴. If 퐴 is diagonalizable, you’ll wind up with 푁 = 0. (6) Compute 푒푡퐷 by ⎡푒휇1푡 ⎤ ⎢ 푒휇2푡 ⎥ 푒푡퐷 = ⎢ ⎥ ⎢ .. ⎥ ⎣ . ⎦ 푒휇푛푡 (7) Compute 푒푡푆 by 푒푡푆 = 푃 푒푡퐷푃 −1 (8) Compute 푒푡푁 by the method for nilpotent matrices discussed above. (9) Compute 푒푡퐴 by 푒푡퐴 = 푒푡푆푒푡푁 .

Department of Mathematics and Statistics, Texas Tech University, Lubbock, TX 79409-1042 E-mail address: [email protected]