Chapter 2 Linear Maps and Odes

Chapter 2

Linear maps and ODEs

The aim of this chapter is to study linear dynamical systems and in particular – to obtain insight in the qualitative features of their phase portrait and how these relate to eigenvalues of the deﬁning matrices; – to derive estimates that settle the stability issue for the zero steady state; – to discuss the more subtle issue of topological equivalence (conjugacy) of these systems.

2.1 Linear maps in Rn

Consider a linear map x Ax, x Rn, (2.1) 7→ ∈ where A =(aij)i,j=1,...,n is an n n real matrix and Ax is the product of the matrix × T A and the vector x =(x1, x2,...,xn) :

(Ax)i = aijxj, i =1,...,n. j=1 X The map (2.1) will be denoted by A as well. Note however, that if another basis in Rn is selected, the same map will be represented by a diﬀerent matrix. We extend (2.1) to complex vectors by the rule

A(u + iv)= Au + iAv, u + iv Cn, u,v Rn. ∈ ∈ Now consider a discrete-time dynamical system N, Rn, Ak or Z, Rn, Ak deﬁned by iteration of the map A (and – if deﬁned – its{ inverse). } { }

2.1.1 Dynamics in eigenspaces Recall that an eigenvalue of A is a number λ C satisfying ∈ Av = λv,

37 38 CHAPTER 2. LINEAR MAPS AND ODES for a nonzero vector v Cn, which is called a corresponding eigenvector. The eigenvalues of a real matrix∈ A can be complex but then occur in complex-conjugate pairs. The set of all eigenvalues of A is denoted by σ(A) and is called the spectrum of A. It should be known to the reader that λ C is an eigenvalue of A if and only ∈ if it is a root of the characteristic polynomial

h(λ) = det(λI A), (2.2) − where I is the unit n n matrix. The polynomial h and hence the characteristic equation × h(λ)=0 (2.3) are independent of the choice of the basis, so one can speak about the eigenvalues of a linear map A. Note that h(A)=0(Hamilton-Cayley Theorem). First consider algebraically simple eigenvalues of A, i.e. simple roots of the characteristic polynomial.

(Simple real eigenvalue) Suppose that λ is a simple real eigenvalue of A with the eigenvector v Rn, i.e. Av = λv. The line ∈ X = x Rn : x = sv, s R λ { ∈ ∈ } is invariant under A, since x X implies Ax X . If x = sv, then ∈ λ ∈ λ Ax = A sv = sAv = λsv.

This means that on the line Xλ the dynamics is simply the multiplication by the k number λ in every time step: if x = s0v then A x = skv where sk = λsk−1, so that

k sk = λ s0.

If λ < 1, s 0 as k + , while s + as k if s = 0. In | | | k| → → ∞ | k| → ∞ → −∞ 0 6 contrast, if λ > 1 and s0 = 0 then sk + as k + , while sk 0 as k . Notice| | that if λ>6 0, then the| time-series| → ∞ is monotone,→ ∞ while if| λ<| →0, the → −∞ time-series exhibits oscillation. Also notice that for λ = 0 the decay of sk to zero is instantaneous. Finally, note that we did not use the assumption that λ is simple. The assumption is neverthelass made, because for simple eigenvalues the formulated results provide the whole story, while for multiple eigenvalues they capture only part of it, see below. (Simple complex-conjugate pair of eigenvalues) Suppose λ, λ¯ is a pair of simple complex eigenvalues with eigenvectors v, v¯ Cn, i.e. ∈ Av = λv, Av¯ = λ¯v.¯

(Again, the observations below are correct even if λ and λ¯ are not simple.) Since λ = λ¯ the vectors v andv ¯ are complex-linearly-independent (see Exercise 2.5.2) and the6 plane in Rn n X ¯ = x R : x = zv +¯zv,¯ z C λ,λ { ∈ ∈ } 2.1. LINEAR MAPS IN RN 39 is invariant under A. Indeed, if x = zv +¯zv¯ then Ax = A(zv +¯zv¯)= zAv +¯zAv¯ = λzv + λ¯z¯v.¯ This means that A acts as z z′ = λz, 7→ i.e. as a multiplication by λ C. If λ = ρeiψ and z = reiϕ, then ∈ z′ =(ρr)ei(ψ+ϕ).

The motion in Xλ,λ¯ is therefore a superposition of the rotation through the angle ψ = argλ and the radial expansion (ρ > 1) or contraction (ρ < 1) with factor ρ = λ . Alternatively| | (but equivalently) we can work with real coordinates (a, b) and write n X ¯ = x R : x = a Re v b Im v, a, b R . λ,λ { ∈ − ∈ } The relationship between the coordinates is given by 1 z = (a + ib). 2 Writing Av = λv as A(Re v + i Im v) = (Re λ + i Im λ)(Re v + i Im v) we obtain A(Re v) = Re λ Re v Im λ Im v, − A(Im v) = Im λ Re v + Re λ Im v.

Therefore, in the coordinates (a, b) the map A restricted to Xλ,λ¯ is represented by the matrix Re λ Im λ cos ψ sin ψ = ρ . Im λ −Re λ sin ψ −cos ψ Whenever the characteristic polynomial (2.2) has n distinct (and therefore necessarily simple) roots λi, i = 1,...,n, we can combine our understanding of the dynamics in each of the invariant subspaces Xλi (for real λi) or Xλi,λ¯i (for complex n λi) to obtain an overall description of the dynamics. The key point is that R can be decomposed into these subspaces.

Deﬁnition 2.1 Let X1,...,Xr be linear subspaces of a linear space X. We say that X is the direct sum of X1,...,Xr, and write X = X X X , 1 ⊕ 2 ⊕···⊕ r whenever every x X can be written in a unique way as ∈ x = x + x + + x 1 2 ··· r with x X for i =1, 2,...,r. i ∈ i 40 CHAPTER 2. LINEAR MAPS AND ODES

As our building blocks Xλi and Xλi,λ¯i are linearly independent and together have dimension n, they form a direct sum decomposition of Rn. So an arbitrary x Rn ∈ can be uniquely written as a linear combination of elements of Xλi and Xλi,λ¯i . The action of A on x can then be simply reconstructed from that on the invariant subspaces by superposition. Since a generic matrix A has only simple eigenvalues, the above consideration completely describes the dynamics generated by such a map. If there are no eigenvalues with λ = 1, we get exponential contraction/expansion | | on the linear eigenspaces, combined with rotation in the case of nonreal eigenvalues. What happens when the characteristic equation (2.3) has multiple roots? (Real multiple eigenvalue) First assume that the multiple (i.e., not simple) eigenvalue λ is real. The preceding analysis still applies when there are n linearly- independent eigenvectors. But this is an exceptional case. Generically, there are m 1 Jordan chains of linearly-independent vectors v(j,k) Rn such that ≥ ∈ Av(j,1) = λv(j,1), Av(j,2) = λv(j,2) + v(j,1),  (2.4)  ···  Av(j,nj ) = λv(j,nj ) + v(j,nj−1),  for j = 1, 2,...,m. The integer nj is called the length of the Jordan chain. The number of Jordan chains with length l can be computed by the formula

N(l,λ)= R 2R + R , l+1 − l l−1 where R = n and R = rank (A λI)k for k 1. The vectors v(j,1) are the eigen- 0 k − ≥ vectors of A corresponding to the eigenvalue λ. The vectors v(j,2), v(j,3),...,v(j,nj) are called generalized eigenvectors corresponding to the eigenvalue λ. It follows from (2.4) that these vectors are null-vectors of the matrix (A λI)nj . n − We now have an nj-dimensional subspace of R deﬁned by

nj X(j) = x Rn : x = c(j)v(j,k), c(j) R (2.5) λ { ∈ k k ∈ } Xk=1 T Rnj that is invariant under (2.1). A coordinate vector c = (c1,c2,...,cnj ) is mapped to ∈ c′ = Jc, where the n n matrix J is given by the Jordan block j × j λ 1 0 λ 1 J =   . (2.6)  λ 1     0 λ      (j) k Therefore, to understand how orbits of (2.1) behave in Xλ , we must consider J . 2.1. LINEAR MAPS IN RN 41

The low-dimensional examples (see Exercise 2.5.8 for the general case) λ 1 k λk kλk−1 = 0 λ 0 λk and k k k−1 k(k−1) k−2 λ 1 0 λ kλ 2 λ 0 λ 1 = 0 λk kλk−1     0 0 λ 0 0 λk clearly indicate that all elements ofJ k tend to zero as k + when λ < 1, whereas some elements of J k diverge as k + when λ →> 1.∞ Moreover,| | in the → ∞ | | critical case λ = 1 only higher multiplicity can and, for suitable initial vectors, will lead to divergent| | orbits. (j) The direct sum of all linear subspaces Xλ , X = X(1) X(2) X(m), λ λ ⊕ λ ⊕···⊕ λ is an invariant linear subspace of Rn called the generalized eigenspace of A corresponding to the eigenvalue λ. Its dimension is (n1 + + nm). Note that Xλ is the null-space of (A λI)l for l = max n . ··· − 1≤j≤m j (Complex multiple eigenvalue) A similar construction applies when λ is a multiple (j,k) complex eigenvalue. In this case, v , j = 1, 2,...,m, k = 1, 2,...,nj, are complex vectors and the Jordan chains corresponding to the eigenvalue λ¯ can be composed of complex-conjugate vectorsv ¯(j,k). Then

nj X(j) = x Rn : x = c(j)v(j,k) +¯c(j)v¯(j,k) , c(j) C (2.7) λ,λ¯ { ∈ k k k ∈ } Xk=1 is a real 2n -dimensional invariant subspace of Rn parameterized by c Cnj . Also j ∈ in this case, J kc 0 as k + when λ < 1, and J kc as k + k k → → ∞ | | k k→∞ → ∞ when λ > 1 and c = 0. The direct sum of all linear subspaces X(j) , | | 6 λ,λ¯ (1) (2) (m) X ¯ = X X X , λ,λ λ,λ¯ ⊕ λ,λ¯ ⊕···⊕ λ,λ¯ is an invariant linear subspace of Rn called the generalized eigenspace of A corresponding to the eigenvalues λ, λ¯. Its dimension is 2(n1 + + nm). It is also possible to deﬁne ···

nj X(j) = x Cn : x = c(j)v(j,k), c(j) C λ { ∈ k k ∈ } Xk=1 and consider the generalized eigenspace corresponding to λ X = X(1) X(2) X(m) λ λ ⊕ λ ⊕···⊕ λ n n as a complex-linear subspace of C . Then X ¯ =(X X¯) R . λ,λ λ ⊕ λ ∩ Since Rn can be decomposed into the direct sum of all generalized eigenspaces ¯ Xλ for real eigenvalues λ and Xλ,λ¯ for complex-conjugated eigenvalue pairs λ, λ, we may summarize the main results of this section as follows. 42 CHAPTER 2. LINEAR MAPS AND ODES

Theorem 2.2 If all eigenvalues of A are located strictly inside the unit circle in the complex plane, then for any x Rn we have that ∈ Akx 0, k k→ as k + . If→ there∞ is an eigenvalue outside the unit circle, or at least one generalized eigenvector corresponding to some eigenvalue on the unit circle, then there exists x Rn such that ∈ Akx , k k→∞ as k + . 2 → ∞ In the next section we shall prove a somewhat stronger result using more general techniques. Moreover, each statement of the above theorem has a suitable converse, see Exercise 2.5.10.

Example 2.3 (Dynamics generated by planar linear maps) When n = 2, equation (2.1) takes the form

x1 a11 a12 x1 x2 7→ a21 a22 x2 and deﬁnes a linear dynamical system N, R2, Ak , where { } a a A = 11 12 (2.8) a a 21 22 Let τ = Tr(A)= a + a , ∆ = det(A)= a a a a . Then the characteristic 11 22 11 22 − 21 12 ∆

0 1 +1 τ −

1 −

Figure 2.1: Stability triangle for planar linear maps. equation (2.3) can be written as

λ2 τλ +∆=0 − 2.1. LINEAR MAPS IN RN 43 and has two (possibly complex) solutions

τ τ 2 λ1,2 = ∆. 2 ± r 4 −

It can easily be shown that both eigenvalues λj satisfy λ < 1 if and only if the point (τ, ∆) falls inside the triangle | |

(τ, ∆) : ∆ < 1, ∆ > τ 1, ∆ > τ 1 { − − − } depicted in Figure 2.1. In this case, in accordance with Theorem 2.2, Akx 0 k k → for all x R2. The reader∈ is also invited to verify that A has – an eigenvalue λ = 1 along the boundary where ∆ = τ 1; 1 − – an eigenvalue λ1 = 1 along the boundary where ∆ = τ 1; −±iθ 1 − − 3 – eigenvalues λ1,2 = e with cos θ = 2 τ along the boundary where ∆ = 1.

2.1.2 Growth estimates and the spectral radius For any norm in Rn we define the associated operator norm for linear maps: Ax A = sup k k = sup Ax . (2.9) k k x k k x6=0 k k kxk=1 The number A describes a rate of expansion/contraction of the map A that de- pends on thek choicek of the norm in Rn. A norm independent measure is given by the spectral radius. Definition 2.4 The spectral radius of a linear map A is defined by

r(A) = sup λ . λ∈σ(A) | |

The relation between these quantities is speciﬁed by the following lemma, which once again shows that the eigenvalues of A yield information about the growth or decay of the time-series obtained by iterating A.

Lemma 2.5 (Gelfand’s formula)

r(A) = lim Ak 1/k = inf Ak 1/k. (2.10) k→∞ k k k≥1 k k Proof: Deﬁne r = inf Ak 1/k. k≥1 k k Then for all ε> 0 there exists m = m(ε) such that Am 1/m r + ε. For arbitrary k, put k = pm + q with 0 q m 1. Then k k ≤ ≤ ≤ − Ak 1/k Am p/k A q/k (r + ε)pm/k A q/k. k k ≤k k k k ≤ k k 44 CHAPTER 2. LINEAR MAPS AND ODES

As k , necessarily q/k 0 and pm/k 1, hence →∞ → → lim sup Ak 1/k r + ε. k→∞ k k ≤ By letting ε 0 we deduce that ↓ lim sup Ak 1/k r = inf Ak 1/k, k→∞ k k ≤ k≥1 k k from which we conclude that the limit of Ak 1/k for k exists and that this limit equals the infimum. k k → ∞ For λ >r the series | | ∞ λ−1 λ−kAk (2.11) Xk=0 converges absolutely. If we multiply (2.11), either from the left or from the right, by (λI A), we obtain − ∞ ∞ λ−kAk λ−(k+1)Ak+1 = λ0A0 = I. − Xk=0 Xk=0 Thus the series (2.11) defines the inverse (λI A)−1 and, accordingly, λ / σ(A). It − ∈ follows that r(A) r. It remains to exclude≤ the possibility that r(A) 0. It follows that Ak (r(A)+ ε)k sup f(z) k k ≤ z∈γ k k and hence r = lim Ak 1/k r(A)+ ε . k→∞ k k ≤ By letting ε 0, we obtain that r r(A). 2 ↓ ≤ Remark: The matrix-valued function λ (λI A)−1 is called the resolvent of A. 7→ − The series representation above is the Taylor series at λ = . By working with z = λ−1 we avoid having to deal with the point at infinity. 3∞

Theorem 2.6 For every ρ>r(A) there exists M 1 such that ≥ Ak Mρk k k ≤ for all k 1. ≥ 2.1. LINEAR MAPS IN RN 45

Proof: If ρ>r(A) then Lemma 2.5 implies Ak ρk for large k and consequently k k ≤ M = sup ρ−k Ak < . k≥0 k k ∞

Thus for all k 0 the inequality ≥ Ak Mρk k k ≤ holds. For k = 0 this gives M 1, since A0 = I and I = 1. 2 Using (2.9), we ﬁnd that ≥ k k

Akx Mρk x , x Rn,k N. (2.12) k k ≤ k k ∈ ∈ This estimate establishes the global asymptotic stability of the origin when r(A) < 1. Indeed, in this case we can choose ρ such that r(A) <ρ< 1. Then for any point x Rn with x ε/M the estimate Akx ρkε holds, which implies the (global) asymptotic∈ stabilityk k ≤ of x = 0. In Exercisek k 2.5.12 ≤ the reader is asked to explore the geometric manifestation of the fact that, possibly, Mρ > 1. We now show that one can introduce an equivalent norm1, tailor-made for the linear map A, such that M reduces to one (the advantage being that the estimate for k = 1 carries all information, in the sense that the estimate for arbitrary k > 1 is obtained by iteration of the one for k = 1).

n Theorem 2.7 Let ρ>r(A). There exists an equivalent norm 1 on R such that A ρ. k·k k k1 ≤ Proof: Deﬁne for x Rn by the formula: k·k1 ∈ ∞ x = ρ−k Akx . k k1 k k Xk=0 Formula (2.10) implies that this series converges. Indeed, for k suﬃciently large and some q < 1, ρ−1 Ak 1/k q k k ≤ and hence ρ−k Akx ρ−k Ak x x qk . k k ≤ k kk k≤k k n Clearly, x 1 0 for all x R and x 1 = 0 if and only if x = 0. Likewise the k k ≥ ∈ k k n properties αx 1 = α x 1 and x + y 1 x 1 + y 1 for x, y R and α R follow straightawayk k from| |k thek correspondingk k ≤ properties k k k ofk . So ∈ is a norm∈ on k·k k·k1 Rn. Moreover, we have

∞ k x x ρ−1 Ak 1/k x , k k≤k k1 ≤ k k k k k=0 ! X 1Recall that two norms on a linear space E are called equivalent, if there are two constants, C C > 0, such that for all x E: C x x C x . 2 ≥ 1 ∈ 1k k≤k k1 ≤ 2k k 46 CHAPTER 2. LINEAR MAPS AND ODES i.e. is equivalent to on Rn. Now, for x Rn, k·k1 k·k ∈ ∞ Ax = ρ−k Ak+1x = ρ( x x ) k k1 k k k k1 −k k Xk=0 so that Ax ρ x , x Rn. 2 k k1 ≤ k k1 ∈ Note that by induction the estimate A ρ implies that k k1 ≤ Akx ρk x , x Rn, k k1 ≤ k k1 ∈ for all k 0. ≥ Deﬁnition 2.8 The map A is called a (strict) linear contraction if for some ρ with 0 <ρ< 1 we have Ax ρ x , x Rn. k k1 ≤ k k1 ∈ Since a linear contraction obviously has the origin as its asymptotically stable ﬁxed point, we have proved the following criterion, which we have already mentioned before.

Theorem 2.9 If all eigenvalues of a matrix A are located strictly inside the unit circle, then A deﬁnes a linear contraction with respect to an appropriate norm and the origin is globally asymptotically stable. 2

As with Theorem 2.2 a certain converse of the statements above holds (see Exercise 2.5.10). Since σ(A−1) = λ−1 : λ σ(A) whenever 0 / σ(A), we can obtain similar { ∈ } ∈ bounds for A−1 in terms of inf λ . λ∈σ(A) | | Theorem 2.10 Assume that λ > ρ>˜ 0 for all λ σ(A). Then there exists M 1 such that | | ∈ ≥ A−k Mρ˜−k, k 0. k k ≤ ≥ Moreover, there exists an equivalent norm on Rn such that k·k1 A−1 ρ˜−1 k k1 ≤ and hence A−k ρ˜−k for all k 0. k k1 ≤ ≥ 2.1.3 Hyperbolic linear maps and spectral projectors In this section we focus on the situation when

σ(A)= σ (A) σ (A), s ∪ u where σs(A) is located strictly inside the unit circle but does not contain zero, while σu(A) is located strictly outside the unit circle. 2.1. LINEAR MAPS IN RN 47

Deﬁnition 2.11 A linear map x Ax in Rn is called hyperbolic if A is invertible and has no eigenvalue λ with λ =17→ . | |

The direct sum of all generalized eigenspaces of A corresponding to σs(A) form an invariant subspace T s of A, called the stable eigenspace, while the direct sum of u all generalized eigenspaces of A corresponding to σu form the unstable eigenspace T of A. Moreover, we have T s T u = Rn, meaning that any x Rn can be uniquely decomposed as ⊕ ∈ x = xs + xu s u with x T and x T . Clearly, σ(A s )= σ (A) and σ(A u )= σ (A). s ∈ u ∈ |T s |T u There are explicit methods to ﬁnd xs (and hence xu) for a given x, using the so called spectral projectors. We begin by assuming that σs(A) consists of just one simple eigenvalue λ R with λ < 1. Let v Rn be the corresponding eigenvector, so that ∈ | | ∈ Av = λv. Let AT denote as usual the transposed matrix corresponding to A. It also has a simple eigenvalue λ with some eigenvector w Rn, ∈ AT w = λw.

Since the eigenvalue λ is simple, the Fredholm decomposition for linear maps (see Lemma 6.14 in Chapter 6) tells us that w, v = 0 and therefore w can be selected h i 6 to satisfy the normalization condition

w, v =1. h i Now the map x P x = w, x v 7→ h i n s n assigns to each x R a vector xs = P x T R . The map P is linear and has obviously the property∈ P 2 = P . We say that∈ P⊂projects Rn onto T s. Note that

T s = x Rn : P x = x = sv : s R . { ∈ } { ∈ } The linear map Q = I P or, in more detail, − x Qx = x w, x v 7→ −h i projects Rn onto T u and

T u = x Rn : P x =0 = x Rn : w, x =0 = w⊥. { ∈ } { ∈ h i }

If σs is composed of ns eigenvalues λ1,λ2,...,λns , all of which are simple and satisfy λ < 1, the corresponding projectors can be defined by | i| ns P = P , Q = I P, i − i=1 X 48 CHAPTER 2. LINEAR MAPS AND ODES where P x = w(i), x v(i), (2.13) i h i Rn is the projection of onto the one-dimensional invariant eigenspace Xλi cor- (i) (i) T (i) (i) responding to the eigenvalue λi. Here Av = λiv and A w = λiw for i =1, 2,...,ns, and 1, i = j, w(i), v(j) = h i 0, i = j. 6 (Verify!) The projectors defined by (2.13) have the following properties: P 2 = P , P P =0, for j = i. (2.14) i i i j 6 Notice that the eigenvalues (and the corresponding eigenvectors) can be complex, provided that , is interpreted as the standard scalar product in Cn. h· ·i There is an alternative method to construct a projector Pi onto the generalized eigenspace corresponding to an eigenvalue λi C, which is applicable for both simple and multiple eigenvalues and does not∈ require computing any eigenvector or generalized eigenvector. In this method, the projection matrix Pi is constructed as a polynomial function of the matrix A. We describe this method briefly below. Actually, we explain how to compute Pi corresponding to any eigenvalue λi of A. Suppose that the characteristic polynomial h(λ) = det(λI A) has d distinct roots: λ ,λ ,...,λ C. Then, it can be written as − 1 2 d ∈ d h(λ)= (λ λ )mi , − i i=1 Y where mi are the algebraic multiplicities of the eigenvalues and hence m1 + +md = n. This implies that the following partial fraction decomposition holds: ···

1 d g (λ) = i , (2.15) h(λ) (λ λ )mi i=1 i X − where gi(λ) are polynomials of degree mi 1 which can be found, for example, by the method of unknown coeﬃcients. − Multiplying both sides of (2.15) by h(λ) leads to the equation

1= pi(λ), (2.16) i=1 X where

h(λ)gi(λ) mj pi(λ)= m = gi(λ) (λ λj) . (2.17) (λ λi) i − − Yj6=i Each pi is a polynomial of degree less than n in λ. Moreover, all the polynomials p2 p and p p with j = i are divisible by the characteristic polynomial h. Indeed, i − i i j 6 2 gi(λ)gj(λ)h (λ) mk pi(λ)pj(λ)= m m = h(λ) gi(λ)gj(λ) (λ λk) . (λ λi) i (λ λj) j − ! − − k6=Yi,k6=j 2.1. LINEAR MAPS IN RN 49

Moreover,

p2(λ) p (λ)= p (λ)(p (λ) 1) = p (λ) p (λ)= p (λ)p (λ), i − i i i − − i j − i j Xj6=i Xj6=i with each term in the sum divisible by h due to the previous formula. Deﬁne now Pi = pi(A), i =1, 2,...,d. (2.18)

Since h(A) = 0 by the Hamilton-Cayley Theorem, we see that the linear maps Pi thus constructed satisfy (2.14), i.e., are projectors. Because each Pi is a polynomial n function of A, it is linear and commutes with A. This implies that each range Pi(C ) is an invariant subspace of A. The formula (2.17) implies that Pi(A)x =0if Ax = λj with j = i. Therefore, since d P = I, P must project Cn onto the generalized 6 i=1 i i eigenspace of A corresponding to the eigenvalue λi: P Cn Xλi = Pi( ).

If there are d distinct eigenvalues λ ,λ ,...,λ satisfying λ < 1, then the pro- s 1 2 ds | i| jectors onto T s and T u are given by

ds P = P and Q = I P, i − i=1 X repectively.

Example 2.12 (Practical computation of spectral projectors) Consider 1 3 A = −3 −7 . 2 2 Then λ +1 3 λI A = − 3 λ 7 − 2 − 2 so that the characteristic polynomial is

h(λ) = det(λI A)= λ2 5 λ +1=(λ 1 )(λ 2) − − 2 − 2 − 1 and the eigenvalues of A are λ1 = 2 and λ2 = 2. Hence A is hyperbolic. The vector 2 v = 1 − 1 is an eigenvector corresponding to the eigenvalue λ1 = 2 , while the vector

1 u = −1 50 CHAPTER 2. LINEAR MAPS AND ODES

is an eigenvector corresponding to the eigenvalue λ2 = 2. The transposed matrix 1 3 AT = − 2 3 7 ! − 2 has 1 w = 1 1 as an eigenvector corresponding to its eigenvalue λ1 = 2 . Notice that w, v =1, w,u =0. h i h i For any vector x x = 1 R2, x2 ∈ we have w, x = x + x and, therefore, h i 1 2 2x +2x P x = w, x v = 1 2 . h i x1 x2 − − This gives 2 2 P = (2.19) 1 1 − − which is indeed a projection matrix: 2 2 2 2 2 2 P 2 = = = P. 1 1 1 1 1 1 − − − − − − The map x P x projects the space R2 onto the eigenline T s spanned by the eigenvector v7→. The matrix 1 2 Q = I P = − − − 1 2 deﬁnes a projection x Qx onto the eigenline T u spanned by the eigenvector u. 7→ Clearly, P Q = QP = 0. To ﬁnd the spectral projectors P and Q using the partial fraction decomposition (2.15), write 1 1 2 2 = = − 3 + 3 . h(λ) (λ 1 )(λ 2) λ 1 λ 2 − 2 − − 2 − This gives 1 = p1(λ)+ p2(λ), where p (λ)= 2 (λ 2), p (λ)= 2 (λ 1 ). 1 − 3 − 2 3 − 2 Therefore

3 3 2 2 P = p (A)= 2 (A 2 I)= 2 − − = , 1 − 3 − − 3 3 3 1 1 2 2 ! − − 2.1. LINEAR MAPS IN RN 51 which (re-assuringly) is the same projector (2.19) as before. For the projector Q, we have

3 3 2 1 2 2 1 2 Q = p2(A)= (A I)= − − = − − . 3 − 2 3 3 3 1 2 2 !

Note that P + Q = I, so that the computation of Q as p2(A) is redundant: It can be found by merely evaluating I P . 3 − Remark: There is another alternative formula for the projector Pi, namely: 1 P = (λI A)−1 dλ. (2.20) i 2πi − Zγi Here the integral is interpreted (for each matrix element) as the contour integral in the complex plane along the small circle γi around λi. It can be computed by the method of residues. Notice that the formula (2.20) is valid in case of multiple eigenvalues as well. Moreover, by a suitable choice of contour γ it works for any spectral set, i.e. any subset of σ(A). For example, γ = S1, where S1 is the unit circle, gives the projector onto T s of a hyperbolic map A. The formula (2.20) can also be generalized to an operator A in a Banach space. 3

Theorem 2.13 For any hyperbolic map A, there is an equivalent norm 1 in −1 k·k which A T s and A T u become linear contractions, i.e. for some ρ with 0 <ρ< 1 we have | | Ax ρ x , x T s, (2.21) k k1 ≤ k k1 ∈ and A−1x ρ x , x T u. (2.22) k k1 ≤ k k1 ∈ Proof: Take some ρ such that 0 r(A T s ) <ρ< 1. According to Theorem 2.7, there ≤ | s exists an equivalent norm 1 on T such that (2.21) holds. −1 k·k −1 Next note that A T u is deﬁned and that σ(A T u ) is contained strictly inside the unit circle. Therefore,| it follows from Theorem| 2.10 that there exists an equivalent norm on T u such that k·k1 A−1x ρ˜ x , x T u, k k1 ≤ k k1 ∈ for someρ< ˜ 1. To obtain the estimates (2.21) and (2.22) with the same ρ, we can simply take ρ := max ρ, ρ˜ . { } Finally, extend the norm to all of Rn by setting

x = x + x for x = x + x T s T u. 2 k k1 k sk1 k uk1 s u ∈ ⊕ Remarks: 52 CHAPTER 2. LINEAR MAPS AND ODES

(1) Note that, by induction, Akx ρk x , x T s, k k1 ≤ k k1 ∈ and A−kx ρk x , x T u. k k1 ≤ k k1 ∈ (2) Note that we did not use in the proof the invertibility of A, i.e. the absence of eigenvalue 0. All that matters for the estimates is that there are no eigenvalues −1 −1 on the unit circle. Indeed, we only used A u and we can replace this by (A u ) |T |T since 0 / σ(A u ). 3 ∈ |T Due to Theorem 2.13 we can give an alternative definition of a hyperbolic linear map, which does not use eigenvalues. Definition 2.14 An invertible linear continuous map A : X X on a Banach space is called hyperbolic if X = T s T u, where both T s and→ T u are invariant ⊕ under A, and with respect to an equivalent norm we have k·k1 −1 A s ρ, A u ρ, k |T k1 ≤ k |T k1 ≤ for some 0 <ρ< 1. Remark: Not all authors include “invertibility” in the definition of “hyperbolicity”. We do this to facilitate the formulation of a topological equivalence result ahead (see Theorem 2.30). For linear maps, the sign of an eigenvalue has no impact on growth or decay of the image length, but it does have an impact on orientation. In particular, the border between orientation preserving and orientation reversing linear contractions corresponds to zero eigenvalue. 3

2.2 Linear autonomous systems of ODEs

Consider an autonomous system of linear ODEs x˙ = Ax, x Rn. (2.23) ∈ The associated ﬂow is given by ϕt(x)= etAx where ∞ tk etA = Ak. k! Xk=0 (Note in particular the group property e(t+s)A = etAesA.) Now consider the linear continuous-time dynamical system R, Rn,ϕt . { } For the spectra we have σ(etA)= etσ(A) = µ C : µ = etλ, λ σ(A) , { ∈ ∈ } for any t R, so in particular zero is never an eigenvalue of etA (indeed, etA has inverse e−∈tA). Note that z etz maps the imaginary axis onto the unit circle and the left half plane inside the7→ unit circle. Accordingly, the condition λ < 1 translates in the | | condition Re λ< 0. 2.2. LINEAR AUTONOMOUS SYSTEMS OF ODES 53

2.2.1 Dynamics in the eigenspaces First consider the dynamics in one- and two-dimensional invariant eigenspaces associated to, respectively, a simple real eigenvalue and a simple complex pair of eigenvalues. (Simple real eigenvalue) Let λ be a simple real eigenvalue of A with eigenvector v Rn, i.e. Av = λv. The line ∈ X = x Rn : x = sv, s R λ { ∈ ∈ } is invariant with respect to the flow, since x Xλ impliesx ˙ Xλ. The function x(t)= s(t)v X satisfies the equation (2.23)∈ if and only if ∈ ∈ λ s˙ = λs, (2.24) since sv˙ =x ˙ = Ax = Asv = sAv = λsv. Thus we find that λt s(t)= s0e and that s(t) 0 as t + if λ< 0, while s(t) 0 as t if λ> 0. → → ∞ → → −∞ (Simple complex-conjugate pair of eigenvalues) Let λ, λ¯ be simple nonreal eigenvalues with the eigenvectors v, v¯ Cn, i.e. ∈ Av = λv, Av¯ = λ¯v.¯

The plane n n X ¯ = x R : x = zv +¯zv,¯ z C R λ,λ { ∈ ∈ } ⊂ is invariant with respect to the ﬂow generated by (2.23). Moreover, x(t) = z(t)v + z¯(t)¯v is a solution if and only if z˙ = λz. This implies λt z(t)= z0e .

iϕ0 If λ = α + iω and z0 = r0e , then

αt i(ϕ0+ωt) z(t)=(r0e )e .

If α = 0, the motion in Xλ,λ¯ is a superposition of rotation (with angular speed ω) and radial6 divergence (if α> 0) or convergence (if α< 0).

If α = Re λ = 0 then the invariant subspace Xλ,λ¯ is ﬁlled with periodic orbits parametrized by r0 > 0. The corresponding periodic solutions are given by

i(ϕ0+ωt) −i(ϕ0+ωt) x(t)= r0 e v + e v¯ , where the phase ϕ0 determines the position of x(0) on the closed orbit. Of course, 2π the period of the orbit is p = ω . 54 CHAPTER 2. LINEAR MAPS AND ODES

Since a generic n n matrix A has n different simple eigenvalues and therefore n linearly independent× eigenvectors, the above consideration completely describes the dynamics of a generic linear system (2.23). If there are no eigenvalues with Re λ = 0, we get exponential contraction/expansion on the linear eigenspaces, combined with rotations in the case of nonreal eigenvalues. There is a possibility, however, that the matrix A has multiple eigenvalues. (Real multiple eigenvalue) Assume first that the eigenvalue λ R. If there is a basis of eigenvectors in the corresponding invariant subspace, then∈ the solutions behave as in the simple real case. Otherwise, there are Jordan chains (2.4) of (j,k) n vectors v R , j = 1, 2,...,m, k = 1, 2,...,nj. The nj-dimensional subspace (j) ∈ X defined by (2.5) is invariant and c =(c ,c ,...,c )T Rnj satisfies λ 1 2 nj ∈ c˙ = Jc, where J is again given by (2.6). Therefore, any solution component is the sum of k λt scalar multiples of t e with k < nj (see Exercise 2.5.8 for an explicit expression for etJ ). (Complex multiple eigenvalue) A similar construction applies when λ C is a ∈ multiple eigenvalue. In this case, v(j,k) are complex vectors and the Jordan chains corresponding to the eigenvalue λ¯ can be composed of complex-conjugate vectors (j,k) v¯ . Then Xλ,λ¯ defined by (2.7) is a real 2nj-dimensional invariant subspace parameterized by c Cnj . All components of Re c(t) and Im c(t) are the sums of ∈k αt k αt scalar multiples of t e sin(ωt), t e cos(ωt), where λ = α + iω, k < nj. Thus, the following theorem holds.

Theorem 2.15 If all eigenvalues λ1,λ2,...,λn of the matrix A satisfy Re λi < 0, then for any x Rn ∈ etAx 0, k k→ as t . If→∞ at least one eigenvalue of A satisﬁes Re λ> 0 or there is at least one generalized eigenvector with corresponding eigenvalue Re λ =0, then there exists x Rn such that ∈ etAx , k k→∞ as t . 2 →∞ Remark: Notice that when x(t) 0 as t , the decay need not to be monotone, see Exercise 2.5.13. Moreover,k k → as in the→ discrete ∞ time case the statements in this theorem can be reversed, see Exercise 2.5.11. 3

Example 2.16 (Phase portraits of planar linear autonomous ODEs) When n = 2, equation (2.23) takes the form

x˙ a a x 1 = 11 12 1 x˙ a a x 2 21 22 2 2.2. LINEAR AUTONOMOUS SYSTEMS OF ODES 55 or x˙ = a x + a x , 1 11 1 12 2 (2.25) x˙ = a x + a x . 2 21 1 22 2 Let τ = Tr(A)= a + a , ∆ = det(A)= a a a a . 11 22 11 22 − 21 12 Then the characteristic equation (2.3) can be written as

λ2 τλ +∆=0 − and has two (possibly complex) solutions

τ τ 2 λ1,2 = ∆. 2 ± r 4 −

If follows right away that both eigenvalues λj satisfy Re λ< 0 if and only if

τ < 0 and ∆ > 0.

Under this condition, Theorem 2.15 implies that etAx 0 as t for any x R2. k k → → ∞ ∈ Traditionally, three generic cases are distinguished, characterized by inequalities in terms of τ and ∆:

s2 x2

v(2)

s x 1 v(1) 1

(a) (b)

Figure 2.2: Canonical (a) and original (b) saddles.

(Saddle: ∆ < 0) The matrix A has one positive and one negative eigenvalue: (1) (2) λ2 < 0 <λ1. Let v and v be the corresponding eigenvectors:

(j) (j) Av = λjv , j =1, 2.

Let us use these eigenvectors as new basis vectors in R2, i.e. write any x R2 in the form ∈ (1) (2) x = s1v + s2v , 56 CHAPTER 2. LINEAR MAPS AND ODES

2 with some s1,2 R. Notice that s1,2 can be considered as new coordinates in R . In these new coordinates,∈ the system (2.25) takes a particularly simple form, namely

s˙ = λ s , 1 1 1 (2.26) s˙ = λ s . 2 2 2 So each of the equations can be solved independently. This gives

λ1t λ2t s1(t)= s1(0)e , s2(t)= s2(0)e , yielding a phase portrait in the (s1,s2)-plane presented in Figure 2.2(a). The phase portrait in the (x1, x2)-plane (see Figure 2.2(b)) appears then by applying a linear transformation to the phase portrait in the (s1,s2)-plane. The equilibrium at the origin is a saddle. 2 (Node: τ > 4∆ > 0) The matrix A has either two positive eigenvalues λ1 > λ2 > 0 (when τ > 0) or two negative eigenvalues λ2 < λ1 < 0 (when τ < 0). The analysis here is very similar to that in the saddle case. Using the eigenvectors v(j) as

s2 x2

v(2)

s1 x v(1) 1

(a) (b)

Figure 2.3: Canonical (a) and original (b) stable nodes. new basis vectors in R2, we obtain once again system (2.26) and solve it as above. When τ < 0, this leads to a phase portrait shown in Figure 2.3(a), since both s1(t) and s (t) tend to zero exponentially fast as t . Notice, however, that due to 2 → ∞ the diﬀerence in the convergence rates, a generic orbit of (2.26) tends to the origin horizontally. The phase portrait in the (x1, x2)-plane (see Figure 2.3(b)) is obtained by a linear transformation. The equilibrium in the origin is a stable node. The case τ > 0 gives an unstable node. The corresponding phase portrait can be obtained from that of the stable node by reversing time. (Focus: 0 < τ 2 < 4∆) The matrix A has now two complex-conjugate eigenvalues λ1,2 = α iω with ± τ 1 α = , ω = √4∆ τ 2. 2 2 − The corresponding eigenvectors v(1) and v(2) are also complex but can be selected to be complex-conjugate: v(1) =v ¯(2) = w C2. ∈ 2.2. LINEAR AUTONOMOUS SYSTEMS OF ODES 57

The vector w can be used to introduce a new coordinate z C which speciﬁes a point x R2 according to the formula ∈ ∈ x = zw +¯zw.¯

As we have already seen, this coordinate satisﬁes

z˙ = λz with λ = α + iω, so that αt i(ϕ0+ωt) z(t)= r0e e , where r0 and ϕ0 are speciﬁed by the initial point at t = 0. For s1 = Re z and s2 = Im z, we get oscillatory behaviour superimposed with growth (α> 0) or decay (α< 0): s (t) = r eαt cos(ϕ + ωt), 1 0 0 (2.27) s (t) = r eαt sin(ϕ + ωt). 2 0 0 When τ < 0, we have α< 0 and the phase portrait in the (s1,s2)-plane is as shown

s2 x2

s1 x1

(a) (b)

Figure 2.4: Canonical (a) and original (b) stable foci. in Figure 2.4(a). All orbits spiral to the origin. A linear transformation produces the phase portrait in the (x1, x2)-plane (see Figure 2.4(b)). The equilibrium at the origin is a stable focus. When τ > 0, one obtains an unstable focus. As we have already noted (see Example 1.17 and Exercises 1.5.13 and 1.5.14 in Chapter 1), a node and a focus of the same stability type are topologically equivalent. We return to this issue in Section 2.3.1 ahead. We leave it to the reader to carry out the analysis of all critical cases (deﬁned by at least one equality relation involving τ and/or ∆). Here we only mention the so-called center, which occurs when τ = 0 but ∆ > 0. In this case both eigenvalues of A are purely imaginary: λ = iω with ω = √∆ > 0. From (2.27) it follows 1,2 ± that all solutions of (2.25) are p0-periodic, where 2π p = 0 ω and, therefore, all nonequilibrium orbits of (2.25) are closed (see Figure 2.5). 3 58 CHAPTER 2. LINEAR MAPS AND ODES

s2 x2

x s1 1

(a) (b)

Figure 2.5: Canonical (a) and original (b) centers.

2.2.2 Growth estimates and the spectral bound Motivated by the observations so far, we can provide the following deﬁnition.

Deﬁnition 2.17 The spectral bound of a linear map A is deﬁned by

s(A) = sup Re(λ). λ∈σ(A)

Theorem 2.18 For every ω>s(A) there exists M 1 such that ≥ etA Meωt k k ≤ for t 0. ≥ Proof: Deﬁne B = ωI + A. Since σ(eB) = e−ω+σ(A) and eλ = eRe(λ), we know − | | that all eigenvalues of eB are inside the unit circle. Let ρ be such that r(eB) <ρ< 1. By Theorem 2.7 we know that

∞ x = ρ−k ekBx k k1 k k Xk=0 B deﬁnes an equivalent norm such that e 1 ρ. We claim that, for some M˜ 1, thek inequalityk ≤ ≥ etB ρtM˜ k k1 ≤ holds for all t 0. Indeed, write t = m ν with m N and ν (0, 1] and deﬁne ≥ − ∈ ∈ −sB M˜ = sup e 1. 0

Then 1 M˜ < and, moreover, ≤ ∞ etB = e(m−ν)B emB e−νB ρmM˜ ρm−ν M˜ = ρtM.˜ k k1 k k1 ≤k k1k k1 ≤ ≤ 2.2. LINEAR AUTONOMOUS SYSTEMS OF ODES 59

It follows that etA e(ω+ln ρ)tM˜ eωtM˜ k k1 ≤ ≤ (here we use that ln ρ < 0). Finally, the equivalence of and yields the k·k1 k·k estimate for etA with an adjusted constant M. 2 k k Like in the discrete time case, we can eliminate the constant M by modifying the norm on Rn such that it is tailor-made for A.

n Theorem 2.19 Let ω>s(A). There exists an equivalent norm 2 on R such that k·k etA eωt k k2 ≤ for all t 0. ≥ Proof: As in the proof of Theorem 2.18, let

∞ x = ρ−k ekBx k k1 k k Xk=0 where B = ωI + A. Now deﬁne − ∞ x = eτBx dτ. k k2 k k1 Z0 The integral converges and

∞ ∞ ∞ M x = eτBx dτ eτB x dτ M ρτ dτ x = x , k k2 k k1 ≤ k k1k k1 ≤ k k1 −ln(ρ)k k1 Z0 Z0 Z0 i.e. M x x . k k2 ≤ −ln(ρ)k k1 Moreover, since x = e−tBetB x, we have

x e−tB etBx etkBk1 etBx k k1 ≤k k1k k1 ≤ k k1 and hence etBx e−tkBk1 x . k k1 ≥ k k1 It follows that ∞ 1 x e−τkBk1 dτ x = x . k k2 ≥ k k1 B k k1 Z0 k k1 Combining the two estimates above, we obtain 1 M x x x , B k k1 ≤k k2 ≤ −ln(ρ)k k1 k k1 meaning that is equivalent to . k·k2 k·k1 60 CHAPTER 2. LINEAR MAPS AND ODES

Finally, we compute ∞ ∞ tA τB tA −ωτ (t+τ)A e x 2 = e e x 1dτ = e e x 1dτ k k 0 k k 0 k k Z ∞ Z ∞ −ω(θ−t) θA ωt −ωθ θA = e e x 1dθ = e e e x 1dθ t k k t k k Z ∞ Z ∞ eωt e−ωθeθAx dθ eωt eθBx dθ = eωt x , ≤ k k1 ≤ k k1 k k2 Z0 Z0 from which it follows that etA eωt for all t 0. 2 k k2 ≤ ≥ n Deﬁnition 2.20 A norm 2 in R is called a Lyapunov norm for a linear system x˙ = Ax if k·k etAx e−αt x , t 0, k k2 ≤ k k2 ≥ for some α> 0.

Theorem 2.21 If all eigenvalues of a matrix A have negative real part, then there n is an equivalent Lyapunov norm 2 on R for the linear system x˙ = Ax and the origin is a globally asymptoticallyk·k stable equilibrium of this system.

Proof: When all eigenvalues of A have negative real part, we have by deﬁnition s(A) < 0. Thus, there exists ω < 0 such that s(A) <ω< 0. Theorem 2.19 implies that is the Lyapunov norm with α = ω > 0. The global asymptotical stability k·k2 − of x = 0 in this case is obvious. 2 As in the discrete time case one can show that stability can be fully characterized in terms of eigenvalue conditions, see Exercise 2.5.11 for details. By time reversal, we obtain the following theorem. Theorem 2.22 Assume that Re(λ) > ω˜ for all λ σ(A). Then there exists M 1 such that ∈ ≥ e−tA Me−ωt˜ , t 0, k k ≤ ≥ and there exists an equivalent norm on Rn such that k·k2 e−tA e−ωt˜ , t 0. k k2 ≤ ≥ 2.2.3 Hyperbolic linear ODEs Deﬁnition 2.23 A linear ODE x˙ = Ax is called hyperbolic if A has no eigenvalue λ with Re λ =0.

The stable and unstable invariant subspaces T s,u of a linear hyperbolic ODE are deﬁned like those of a linear hyperbolic map. Namely, we can write as before

σ(A)= σ (A) σ (A), s ∪ u where now

σ (A)= λ σ(A):Re λ< 0 and σ (A)= λ σ(A):Re λ> 0 . s { ∈ } u { ∈ } 2.3. TOPOLOGICAL CLASSIFICATION OF LINEAR SYSTEMS 61

Then T s and T u will be spanned by all eigenvectors and generalized eigenvectors of A corresponding to σs(A) and σu(A), respectively. Spectral projectors can also be computed as before, with γi in the formula (2.20) replaced by any contour encircling σs(A). The dynamics on T s is described by Theorem 2.21. There is no need to study the dynamics on T u separately, since the results can be obtained from those about T s by reversal of time, i.e. the substitution t t. Thus, we arrive at the following 7→ − theorem (we leave it to the reader to provide the details of the proof).

Theorem 2.24 For any hyperbolic system x˙ = Ax, there is an equivalent norm on Rn such that for some α> 0 we have for all t 0 k·k2 ≥ etAx e−αt x , x T s, k k2 ≤ k k2 ∈ and e−tAx e−αt x , x T u. 2 k k2 ≤ k k2 ∈ 2.3 Topological classiﬁcation of linear systems

2.3.1 Topological equivalence of hyperbolic linear ODEs We consider ﬁrst the problem of topological classiﬁcation of linear ODEs. We advise the reader to make Exercises 1.5.13 and 1.5.14 before proceeding.

Theorem 2.25 The linear system

x˙ = Ax, x Rn, (2.28) ∈ where all eigenvalues of A have negative real part, is topologically conjugate to the system y˙ = y, y Rn. (2.29) − ∈ tA −αt Proof: Let 2 be a Lyapunov norm: e x 2 e x 2, α> 0. Any nontrivial orbit of (2.28)k·k crosses k k ≤ k k Σn−1 = x Rn : x =1 { ∈ k k2 } exactly once. Denote by H : Σn−1 Sn−1 the map that assigns to the point of intersection of the ray sv : s> 0 with→ Σn−1 the point of intersection of the same { } ray with Sn−1 (see Figure 2.6), i.e. x H(x)= x k k where the standard (Euclidian) norm in Rn is used. k·k Deﬁne now the map h : Rn Rn as follows. For any x = 0, let τ(x) =0 be the (positive or negative) time such→ that 6 6

x = eτ(x)Ax˜ withx ˜ Σn−1. ∈ 62 CHAPTER 2. LINEAR MAPS AND ODES

v v y˜ H x˜ y x

n−1 Σ Sn−1

Figure 2.6: Construction of the homeomorphism x y = h(x). 7→

We havex ˜ = e−τ(x)Ax. Mapx ˜ y˜ = H(˜x) and then set 7→ y = h(x)= e−τ(x)y˜ = e−τ(x)H(e−τ(x)Ax).

Let h(0) = 0. The map h : Rn Rn thus constructed has the following properties: → (i) h is a homeomorphism on Rn (a detailed check of this is left to the reader); (ii) h maps orbits of (2.23) onto orbits of (2.29), preserving time parametrization. For the proof of the last statement use the fact that τ(esAx)= τ(x)+ s. 2 Applying Theorem 2.25 in the stable eigenspace directly and after time reversal in the unstable eigenspace, we arrive at the following result. Theorem 2.26 Two hyperbolic linear systems in Rn, x˙ = Ax and y˙ = By, are topologically conjugate if the matrices A and B have the same number of eigenvalues with negative real part. Remark: Actually, the inverse result is also valid, i.e. one can write “if and only if” in Theorem 2.26. However, this fact is only of theoretical interest, since it is rarely known a priori that two linear ODE systems are topologically conjugate. 3

2.3.2 Topological equivalence of hyperbolic linear maps When are two hyperbolic linear maps topologically equivalent ? We begin with an example that illustrates that that we can use essentially the same idea as used in the continuous time case, but, because now we deal with “jumps” rather than with a continuous orbit, we need to replace the boundary of a ball by a set with non-empty interior.

Example 2.27 (Topologically equivalent scalar linear maps) Consider two linear scalar maps: 1 1 f : x x and g : y y. 7→ 2 7→ 3 2.3. TOPOLOGICAL CLASSIFICATION OF LINEAR SYSTEMS 63

x′ y′ x′ = x y′ = y

x′ = 1 x 2 ′ 1 y = 3 y

0 x 0 y 1 1 x 2 1 h(x) 3 1 H

Figure 2.7: The maps f : x 1 x and g : y 1 y are topologically conjugate. 7→ 2 7→ 3

These maps are not smoothly conjugate, since they have diﬀerent eigenvalues, i.e. 1 1 2 and 3 (see Exercise 1.5.12). However, they are topologically conjugate. Indeed, the following construction (see Figure 2.7) gives a conjugating homeomorphism h : R R. → Take any, for example linear, homeomorphism

H : [ 1 , 1] [ 1 , 1], 2 → 3 1 1 satisfying H( 2 )= 3 , H(1) = 1. The intervals are called the fundamental domains. For x (0, 1 ) take ∈ 2 h(x)= gk(x)(H(f −k(x)(x))), where k(x) is the minimal2 positive integer k, such that

f −k(x) [ 1 , 1]. ∈ 2 For x> 1 take h(x)= g−l(x)(H(f l(x)(x))), where l(x) is the minimal positive integer l, such that

f l(x) [ 1 , 1]. ∈ 2 Make the analogous construction for x < 0. Set h(0) = 0. The map h constructed in this manner has the following properties: (i) h : R R is a homeomorphism; if H is linear, h is piecewise-linear. (ii) h sends→ orbits of f to orbits of g, i.e. f = h−1 g h. ◦ ◦ This means that f g. ∼ Now consider another map g defined by 1 g : y y. 7→ −3 2The adjective “minimal” is added to be specific but it is, in fact, superfluous. Ambiguity can only arise when x = 2−n but then both choices k(x) = n 1 and k(x) = n yield h(x)=3−n so, after all, the ambiguity is harmless. − 64 CHAPTER 2. LINEAR MAPS AND ODES

This map is not topologically equivalent to f. Indeed, suppose that there exists a continuous map h : R R sending orbits of f to orbits of g. In particular, it should → map the orbit of f 1 1 ..., 1, , ,... 2 4 to the orbit of g a a . . . , a, , ,..., −3 9 where a = 0. Therefore, we must have 6 1 a 1 a h(1) = a, h = , h = , 2 −3 4 9 showing that h is not monotone. Therefore, h is not invertible. 3

Lemma 2.28 Let A : x Ax, x Rn, (2.30) 7→ ∈ be an invertible linear map which is a contraction with respect to a norm in Rn. Then any linear map B, k·k B : y By, y Rn, (2.31) 7→ ∈ with B A suﬃciently small, is topologically conjugate to A. k − k Proof: We have Ax ρ x , x Rn, k k ≤ k k ∈ for some ρ< 1. Take the unit sphere

Sn−1 = x Rn : x =1 { ∈ k k } and consider its image ASn−1 IntSn−1. Deﬁne a fundamental domain for A: ⊂

v w DA A−1v A−1v A−1v ASn−1 k k

Sn−1

Figure 2.8: Fundamental domain for the contraction A.

D = Sn−1 (Int Sn−1 Int ASn−1). A ∪ \ 2.3. TOPOLOGICAL CLASSIFICATION OF LINEAR SYSTEMS 65

DB DA w H H(w)

−1 A B

−1 id ASn BSn−1

Sn−1 Sn−1

−1 Figure 2.9: Constraints on the map H : H n−1 = id, H n−1 = BA . |S |AS

This is a closed annulus between Sn−1 and ASn−1 (see Figure 2.8). If B A ε with suﬃciently small ε, we obtain by the triangle inequality k − k ≤ B = B A + A B A + A ε + ρ =: ρ < 1 k k k − k≤k − k k k ≤ 1 and therefore By ρ y , y Rn. k k ≤ 1k k ∈ Deﬁne the fundamental domain for B by

D = Sn−1 (Int Sn−1 Int BSn−1). B ∪ \ Suppose that we can construct a homeomorphism

H : D D , A → B which maps the corresponding boundaries of the fundamental domains DA and DB into each other and which “agrees” with the maps A and B on them, in the sense that H n−1 = id. (2.32) |S and −1 H n−1 = BA (2.33) |AS (see Figure 2.9). Then we can deﬁne a map

h : Rn Rn → by setting ﬁrst h = H and then deﬁning for x IntASn−1, x =0, |DA ∈ 6 h(x)= Bk(x)H(A−k(x)x),

−k where k(x) is the minimal integer k > 0, such that A x DA. Similarly define h(x) for x ExtSn−1, and finally set h(0) = 0. A careful∈ reasoning using (2.32) ∈ and (2.33) shows that the resulting map h : Rn Rn is a homeomorphism with the property → Ax = h−1(Bh(x)), x Rn, ∈ 66 CHAPTER 2. LINEAR MAPS AND ODES meaning that B is topologically conjugate to A. Therefore, the lemma will be proved if we construct a homeomorphism H : D D , satisfying (2.32) and (2.33). The A → B rest of the proof is devoted to this construction. First, we notice that any ray x Rn : x = sv, s (0, ) { ∈ ∈ ∞ } for some given v Rn with v = 1, intersects ASn−1 at a unique point ∈ k k A−1v v w = A = A−1v A−1v k k k k (see Figure 2.8). This allows us to introduce new coordinates (v, t) in DA. Namely, identify x D with a point (v, t) Sn−1 [0, 1] as follows. For each x D , take ∈ A ∈ × ∈ A x v = Sn−1, x ∈ k k compute 1 x a = = k k R, A−1v A−1x ∈ k k k k so that w = av, and then set a x t = −k k [0, 1]. a 1 ∈ − (In Exercise 2.5.16 below you are asked to compute the inverse of the map x (v, t).) Make a similar construction for D , identifying each point y D with7→ B ∈ B (η, τ) Sn−1 [0, 1]. The∈ boundary× constraints (2.32) and (2.33) motivate us to define two homeo- morphisms δ : Sn−1 Sn−1, 1,0 → by δ1 = id and BA−1v δ (v)= . 0 BA−1v k k From the fact that B is close to A and, thus, BA−1 is close to the unit matrix, we infer that δ0 is close to the identity map. The map (1 t)δ (v)+ tδ (v) δt(v)= − 0 1 , (1 t)δ (v)+ tδ (v) k − 0 1 k is a homeomorphism on Sn−1 for all t [0, 1] (Prove this!). Therefore, the map ∆ : Sn−1 [0, 1] Sn−1 [0, 1] defined∈ by the formula × → × ∆(v, t)=(δt(v), t) is a homeomorphism. Finally, the homeomorphism ∆ induces, via the above constructed coordinates, a homeomorphism H : D D A → B 2.3. TOPOLOGICAL CLASSIFICATION OF LINEAR SYSTEMS 67 with the required properties (2.32) and (2.33). Indeed, the property (2.32) is obvious, while the property (2.33) can be verified as follows. Take v Sn−1 and consider the ∈ point v w = ASn−1 A−1v ∈ k k n−1 n−1 that has coordinates (v, 0) S [0, 1] in DA. Note that any point in AS can be uniquely represented in∈ this way.× Then

v BA−1v BA−1w = BA−1 = BSn−1. A−1v A−1v ∈ k k k k On the other hand, BA−1v δ (v)= Sn, 0 BA−1v ∈ k k which implies BA−1v η = δ0(v)= δ (v)= Sn. 0 BA−1v ∈ k k The point with coordinates (η, 0) Sn−1 [0, 1] corresponds to the point ∈ × η BA−1v H(w)= = BSn−1, B−1η A−1v ∈ k k k k from which we conclude that H(w)= BA−1w for w ASn−1. 2 ∈ Theorem 2.29 Two linear invertible contractions in Rn that either both preserve or both reverse the orientation of Rn are topologically conjugate.

Proof: Let x Ax and x Bx be two such maps. By the assumption, the 7→ 7→ determinants of the corresponding matrices A and B are both positive or negative. Therefore, there exists a continuous family of matrices M(t), t [0, 1], such that M(0) = A, M(1) = B, and ∈

det M(t) =0, r(M(t)) < 1 6 for all t [0, 1] (see Exercise 2.5.17). By Lemma 2.28, any two linear maps M(t′) and M(t∈′′) are topologically conjugate if t′′ t′ is suﬃciently small. Since [0, 1] can be covered with ﬁnitely many such intervals,| − A| is topologically conjugate to B. 2 Suppose that f : Rn1 Rn1 , g : Rn1 Rn1 , 1 → 1 → −1 are two topologically equivalent maps, i.e. f1 = h1 g1 h1 for some homeomorphism h : Rn1 Rn1 . Suppose also that ◦ ◦ 1 → f : Rn2 Rn2 , g : Rn2 Rn2 , 2 → 2 → 68 CHAPTER 2. LINEAR MAPS AND ODES

−1 are two other topologically equivalent maps, i.e. f2 = h2 g2 h2 for some n2 n2 ◦ ◦ homeomorphism h2 : R R . Then obviously the direct product maps f,g : 1 2 1 2 → Rn Rn Rn Rn deﬁned coordinate-wise, × → × x f (x ) x g (x ) f : 1 1 1 , g : 1 1 1 , x2 7→ f2(x2) x2 7→ g2(x2) are topologically equivalent. Indeed,

f = h−1 g h, ◦ ◦ where h : Rn1 Rn2 Rn1 Rn2 is a homeomorphism deﬁned by × → × x h (x ) h : 1 1 1 . x2 7→ h2(x2) The topology in the product space can be induced, for example, by the norm

x = x + x . k k k 1k k 2k

Let ns(M) and nu(M) denote the number of eigenvalues of the matrix M inside and outside the unit circle, respectively (taking multiplicities into account). Further, let ps(M) and pu(M) denote the products of the eigenvalues inside and outside the unit circle, respectively. Recall that topological conjugacy of two (in our case linear) invertible maps is equivalent to the conjugacy of their inverses (with the same conjugating homeomorphism). Applying Theorem 2.29 to the restriction of a hyperbolic linear map to its stable eigenspace, and to the inverse of the restriction to the unstable eigenspace, we obtain the following theorem.

Theorem 2.30 Two linear hyperbolic maps in Rn, x Ax, y By, are topologically conjugate if the following properties hold simultan7→eously: 7→

ns(A)= ns(B), ps(A)ps(B) > 0, pu(A)pu(B) > 0. 2

Remark: As in the continuous time case, the inverse for Theorem 2.30 is also valid but again of minor interest. 3

2.4 References

There are many good books, where linear autonomous ODEs are treated. The standard references here are [Arnol’d 1973, Hirsch & Smale 1974]. The analysis of linear maps can be done using similar techniques, see, for example [Kato 1980, Chapter I]. The construction of spectral projectors using the partial fraction decomposition is inspired by unpublished notes of Joop Kolk (Utrecht University). The fundamental domains are used in [Katok & Hasselblatt 1995] to prove the local topological equivalence of a map with a stable hyperbolic linear part to its linearization. 2.5. EXERCISES 69 2.5 Exercises

E 2.5.1 (Exceptional cases of real simple eigenvalue)

Describe the dynamics generated by the linear map A on the line Xλ when (i) λ = 1; (ii) λ = 1; − (iii) λ = 0.

E 2.5.2 (Linear algebra refresher)

(a) Let λ be a nonreal eigenvalue of A with eigenvector v. Show that: (i) v andv ¯ are linearly independent in Cn; (ii) Re v and Im v are linearly independent in Rn. (b) Let λ and µ be eigenvalues of A with eigenvectors v and w, respectively. Show that λ = µ implies that these vectors are linearly independent. 6 (i) (i) (i) (c) Let Av = λiv with v = 0 and λi = λj for i = j, where i, j =1, 2,...,n. Prove that the vectors v(i) are linearly independent.6 6 6

E 2.5.3 (Spectral projection)

Consider 11 3 A = 2 6 −7 − 2 as a generator of a discrete-time dynamical system in R2. Compute the projectors corresponding to the decomposition R2 = T s T u. ⊕

E 2.5.4 (Stability conditions for linear maps in R2 and R3)

(a) Prove all statements formulated in Example 2.3. (b) Consider a three-dimensional linear map

x Ax, x R3 7→ ∈ and write the characteristic equation det(λI A) = 0 in the form − 3 2 λ + a0λ + a1λ + a2 =0.

Show that all its roots lie strictly inside the unit circle if and only if the following inequalities hold simultaneously:

1+ a + a + a > 0, 1 a + a a > 0, a < 1, and 1 a2 >a a a . 0 1 2 − 0 1 − 2 | 2| − 2 1 − 0 2 Also verify that λ =1isarootif1+a +a +a = 0, that λ = 1isarootif1 a +a a = 0, 1 0 1 2 1 − − 0 1 − 2 and that there are two roots λ1,2 with λ1λ2 = 1 if

1 a2 = a a a . − 2 1 − 0 2 ±iθ 1 Show that in the latter case λ1,2 = e with 0 <θ<π where cos θ = 2 (a2 a0), provided that a a < 2. − | 2 − 0|

E 2.5.5 (Operator norm of matrices) 70 CHAPTER 2. LINEAR MAPS AND ODES

Let x be the standard Euclidian norm of x Rn. Consider a n n matrix A with real elements andk deﬁnek its operator norm A by the formula∈ (2.9) with Ax understood× as the matrix-vector product. k k

(a) Prove that A = √µmax, where µmax is the largest eigenvalue of the symmetric matrix AT A. (Hint: Ax 2k=k Ax, Ax = AT Ax, x , where x, y = xT y, the standard scalar product of x, y Rn. Recallk thatk ah symmetrici h matrix hasi an orthonormalh i basis consisting of eigenvectors.) (b)∈ Using (a), show that for a (2 2)-matrix × a b A = , a,b,c,d R, c d ∈ the operator norm satisﬁes 1 A 2 = a2 + b2 + c2 + d2 + [(a d)2 + (b + c)2][(a + d)2 + (b c)2] . k k 2 − − h p i (c) Apply the formula derived in (b) to prove that 1 A−1 = A k k det A k k | | for any nonsingular 2 2 real matrix A. Is this result valid for n> 2 ? × E 2.5.6 (Fibonacci numbers) Deﬁne a sequence of integers by the recurrence relation

ak = ak− + ak− , k 2, 1 2 ≥ with initial condition a0 = a1 = 1. Find an explicit expression for ak. Hint: Write a a k = A k−1 , ak ak +1 where 0 1 A = , 1 1 and consider a discrete-time dynamical system generated by the linear map x Ax in R2 (cf. Exercise 1.5.6 in Chapter 1). Find eigenvalues λ and µ and corresponding eigenvectors7→ v and w of A and compute the associated projection operator(s). Answer : 1 k+1 k+1 ak = (1 + √5) (1 √5) . 2k+1√5 − − h i E 2.5.7 (Quantum oscillations) In Quantum Mechanics, a system with two observable states is characterized by a vector

a ψ = 1 C2, a2 ∈ where ak are complex numbers called amplitudes, satisfying the condition

a 2 + a 2 =1. | 1| | 2| 2 The probability of ﬁnding the system in the kth state is equal to pk = ak , k =1, 2. The behaviour of the system between observations is governed by the| |Heisenberg equation: dψ i~ = Hψ, dt 2.5. EXERCISES 71 where the real symmetric matrix

E A H = 0 , E ,A> 0, A− E 0 − 0 is the Hamiltonian matrix of the system and ~ is Planck’s constant divided by 2π. (i) Write the Heisenberg equation as a system of two linear complex diﬀerential equations for the amplitudes. (ii) Integrate the obtained system and show that the probability pk of ﬁnding the system in the kth state oscillates periodically in time. (iii) How does p1 + p2 behave?

E 2.5.8 (Jordan blocks) Write the Jordan block from (2.6) as J = λI + N with the matrix

01 0 0 1   N =  0 1     0 0      (nilpotent since N nj −1 = 0) and use the binomial formula to ﬁnd expressions for J k and exp(tJ)= ∞ (tJ)k k=0 k! .

PE 2.5.9 (Jordan decomposition) It is known3 that each n n complex matrix A can be uniquely written as × A = S + N, where S is diagonal in some basis (semisimple) and N m = 0 for some m

d m− 1 tj exp(tS)= eλitP , and exp(tN)= N j . i j! i=1 j=1 X X (iv) Show that d m−1 j λit t j exp(tA)= e N Pi.  j!  i=1 j=1 X X (v) Combine the formula from step (iv) with the partial fraction decomposition method to compute Pi and obtain an efficient algorithm for finding flows of linear autonomous systems.

E 2.5.10 (Stability and eigenvalues for maps)

3See, for example, Section 6.2 in Hirsch, M.W. and Smale, S. Diﬀerential Equations, Dynamical Systems, and Linear Algebra. Academic Press, New York-London, 1974. 72 CHAPTER 2. LINEAR MAPS AND ODES

(i) Extend Theorem 2.9 by showing that asymptotic stability of the origin implies that all eigenvalues of A lie strictly inside the unit circle. (ii) Prove that for a map x Ax the following are equivalent (a) The origin is stable; 7→ (b) Each eigenvalue either lies strictly inside the unit circle or lies on the unit circle but has no generalized eigenvector; n (c) There exists a norm 1 in R such that A 1 1. Hint: (a) (b) followsk·k from Theorem 2.2 andk k for≤ (b) (c) prove the boundedness of Ak(k 0) and deﬁne x⇒ = sup Akx . ⇒ ≥ k k1 k≥0 k k E 2.5.11 (Stability and eigenvalues for ﬂows) Make an analog of Exercise 2.5.10 for ODE’sx ˙ = Ax. (i) Asymptotic stability of the origin implies that all eigenvalues of A have negative real part. (ii)Lyapunov stability holds if and only if all eigenvalues either have negative real part or have zero real part but no generalized eigenvectors. Another equivalent condition says that there exists an n tA equivalent norm 1 in R such that e 1 1 for all t 0. Hint: For thek·k construction of a properk normk ≤ use x =≥ sup etAx . k k1 t≥0 k k E 2.5.12 (Asymptotic stability and contraction) Consider two maps in R2 generated by the matrices

1 0 1 1 A = 2 and B = 2 − 2 . 0 1 1 1 4 4 ! (i) Show that the origin is an asymptotically stable ﬁxed points for both maps. (ii) Which of these maps is a linear contraction with respect to the standard norm x = 2 2 R2 k k x1 + x2 in ? Hint: Consider the images AU and BU of the unit disk p U = x R2 : x2 + x2 1 { ∈ 1 2 ≤ } under the maps A and B, respectively. Show that AU is located strictly inside U, while BU does not have this property (see Figure 2.10). (iii) Construct several ﬁrst elements of the sequences of the images

U, AU, A2U, A3U,... and U,BU,B2U,B3U,... and convince yourself that both sequences contract towards the origin.

E 2.5.13 (Asymptotic stability and monotonicity) Consider a linear ODE x˙ = Ax, x R2, (2.34) ∈ deﬁned by the matrix 2 5 2 A = − 1 2 . − 2 − 5 ! (i) Prove that x = 0 is an asymptotically stable equilibrium of (2.34). What kind of equilibrium is it? T (ii) By numerical integration, show that the orbit starting at x0 = (1, 0) crosses the unit 2 2 2 circle x = x1 + x2 = 1 several times (see Figure 2.11 (a)). Prove this analytically. (iii)k k Deﬁne x 2 = x2 +4x2. Prove that is a norm in R2 that is equivalent to . k k2 1 2 k·k2 k·k 2.5. EXERCISES 73

2 2 y3 y3 x2 x2 1.6 1.6

1.2 1.2 BU

0.8 0.8

0.4 AU 0.4

0 0

-0.4 -0.4

-0.8 -0.8

-1.2 U -1.2 U

-1.6 -1.6

xx3 xx31 -2 1 -2 -2 -1.6 -1.2 -0.8 -0.4 0 0.4 0.8 1.2 1.6 2 -2 -1.6 -1.2 -0.8 -0.4 0 0.4 0.8 1.2 1.6 2 (a) (b)

Figure 2.10: Asymptotically stable ﬁxed points: (a) a linear contraction; (b) not a contraction.

1.5 1.5 y y x2 x2 1.2 1.2

0.9 0.9

0.6 0.6

0.3 0.3

0 0

-0.3 -0.3

-0.6 -0.6

-0.9 -0.9

-1.2 -1.2

x1x x1x -1.5 -1.5 -1.5 -1 -0.5 0 0.5 1 1.5 -1.5 -1 -0.5 0 0.5 1 1.5 (a) (b)

Figure 2.11: Nonmonotone (a) and monotone (b) convergence. 74 CHAPTER 2. LINEAR MAPS AND ODES

(iv) Prove that any “circle” x 2 = R0 > 0 is crossed by any orbit of the system only once (see Figure 2.11 (b)). k k Hint: Show that d 4 x(t) 2 = x(t) 2 dtk k2 −5k k2 along any nonequilibrium solution of (2.34). Find R(t) = x(t) by integrating this diﬀerential k k2 equation and demonstrate that it converges to zero monotonously. Is 2 a Lyapunov norm for (2.34)? k·k

E 2.5.14 (Orientation and phase portraits) In the two dimensional focus case derive a criterion for the matrix coeﬃcients that characterizes whether orbits are rotating clockwise or counterclockwise. Compare with the phase portraits from the previous exercise.

E 2.5.15 (Explicit conjugating homeomorphism) Verify that map h : R R defined by the formula → xν , x 0, h(x)= x ν , x<≥ 0, −| | defines a homeomorphism conjugating the maps f and g from Example 2.27 for a suitable ν (which one ?). E 2.5.16 (Inverse map in Lemma 2.28) Compute the inverse of the map x (v,t) defined in the proof of Lemma 2.28. 7→ Answer: 1 1 (v,t) t 1 + v. 7→ − A−1v A−1v k k k k E 2.5.17 (Connectedness of contractions) Show that the set of contracting and positively oriented matrices M = A : det(A) > 0, r(A) < 1 is connected, i.e. for any two A, B M there is a continuous path M(t{) M, 0 t 1 such that} M(0) = A and M(1) = B. ∈ ∈ ≤ ≤ 1 Hint: It is sufficient to connect any A to the special matrix B = 2 I M. If A has no negative eigenvalues, then take the line tB + (1 t)A. If A has negative eigenvalues,∈ they must occur in pairs. Isolate the invariant subspaces belonging− to them and move the eigenvalues to the positive real axis by a suitable rotation.