Structurally Stable Systems Are Not Dense

STRUCTURALLY STABLE SYSTEMS ARE NOT DENSE.

By S. SMALE.

In this note, we give a negative answer to "the problem of structural stability"; are the structurally stable differential equations dense in the C' topology in all (first order, ordinary, autonomous) differential equations?

MAIN THEOREM. There exists a compact 4 dimensional manifold M, an open set U in the space of Cr vector fields, Cr topology, r > 0, on MI such that no X C U is structurally stable. This problem has been stated explicitly in the above form by the author on several occasions, see e. g. [10] and [11]. However the problem is older, considered for example by Soviet mathematicians, and by Peixoto who gave an affirmative answer [7] for the 2-disk and [8] for compact 2-manifolds. Evidence for a positive answer in higher dimensions was given by the author [12], [13] where it was shown that certain examples of differential equations having an infinite number of periodic solutions were structurally stable, and Anosov [2] gave further examples with the same properties. The notion of structural stability was introduced by Andronov and Pontriagin in 1937 [1] (see also Lefschetz [5]). We recall the definitions now. A differential equation (lst order, ordinary) X, will be a Cr tangent vector field on a C? manifold M, which we will assume compact for purposes of this paper. Of course X, through its solution curves, generates a 1 parameter group of diffeomorphisms ct of M. An equivalence between differential equations X and X' on M is a homeomorphism h: 11M-->M which sends a sensed solution curve of X onto one of X'. Fixing a metric on M, it is an E-equivalence if it is pointwise within e of the identity. The Cr differential equations on M form a normed space with the C' norm. Then X is structurally stable if given E> 0, there exists 8 > 0 such that if 11X-XT Ile < 8, then X' is e-equivalent to X. We will first construct a diffeomorphism g of a 3-dimensional manifold (a 3-torus) and use this to construct a differential equation X0 on a 4-dimensional manifold. This X, has the property that there is a neighborhood N of XOin the C' topology such that no X in N is structurally stable.

Received June 1, 1965. 491 492 S. SMALE.

The starting point is the linear transformation of R3 given by the matrix a b 0\ A= c d 0 0Q O r/ whereB== (a b) has determinant 1, a, b, c, d are integers and 0 < r < l, r small. We assume further that the eigenvalues of B are real and not ? 1. Then B as a transformation of the x-y plane induces a diffeomorphism BO of the 2-torus T. Also A as a transformation on R3 induces a diffeomorphism AO: T X R T X R with T X 0 invariant and all other points of T X R contracting towards T X 0 under AO. Of course Ao restricted to T X 0 is essentially the same as B, on T. We shall now recall some of the structure of this discrete flow (see [2], [3], [10], R. Thom [unpublished]). Let II: R2->T be the covering map, and p ==H(0,O) so that p is a fixed point of BO. Then the two 1-dimensional eigenspaces of B project under II into a stable and unstable manifold through p (see [11] for these definitions of these terms) which we will denote respectively by W' (p) and Wu(p). Both We(p) and W-u(p) are dense in T. The periodic points of Bo are dense in T (this fact may be proved algebraically, one may use the Generalized Birkhoff Theorem of [13] or Anosov [2]), and each of these also has a 1-dimensional stable (and unstable) manifold associated to it. These will be disjoint, parallel (using the Euclidean structure on R2) to W8(p) and each is dense in T. Now we pass to the extension A,: TXR--TXR of BO. It is easy to see that the periodic points of Ao coincide with those of Bo as do also the 1-dimensional unstable manifolds which we will denote from now on by W,u(qi), Xi 1,2, , q, the periodic points of AO. The stable manifolds now however are two dimensional, being in T X R of the form Ws(qj) X R where W8(q) C T is the 1-dimensional stable manifold of B0: T - T for the periodic point qi. From now on we denote this 2-dimensional stable manifold W8(qi) X R by W2 (qO). Thus the stable manifolds of the periodic points of Ao are a countable, dense family of parallel "planes" in T X R. Let AO denote the restriction of AO to T X [-1, C T X R. In a neighborhood S of radius l about the point b (0, 0, 2) of R3, let C be a linear transformation which contracts in the x-y plane through b, leaving b STRUC(TURALLY STABLE SYSTEMS. 493 fixed and leaving the z-axis invariant, expanding on it away from b. Denote by COthe induced transformation defined on the corresponding neighborhood S 0=11(S) of r1l(b) in TX]R, ll: R3--TXR our covering map, and let J { (0, 0, z) E T XR I 0 < z < 2}. Now it is easy to define a diffeomorphism f on a neighborhood of TX [- 1,1]USoUJ in TX]R into TX]R which is A1o on TX [-1,1], COon S0, and leaves J invariant with no fixed points on J, except 0 and II,(b). In fact we may assume f is extended not just to a neighborhood of T X [-1, 1] U So U J, but to a diffeomorphism of the whole compact mani- f fold T X S' > TX S', where S' is [- 3, 3] with end points identified. Our desired diffeomorphism g is obtained by perturbing f in a neighborhood G of a point a in T X S' as follows. We assume r =- . Let ao -II (0, 0, i) and Go III,(Ni/8 (0, 0, i), N6(x) being the neighborhood of x of radius e. Then let a f-1(Qox) and G=f-1(Go) so since r f(G) nG =--=. Using the obvious linear structure on G, let g f outside G and on G let g =f + -cph where h is the vector (1, 0, 0), ( is a non-negative Coo function with compact support in G and non-degenerate maximum value 1 at X and finally -1> 0 is small enough so that g is a diffeomorphism. Note h is transversal to the "planes" W28(qi). We now show that if g': M -> M is any diffeornorphismsufficiently close to g in the C' topology, then there exists a diffeomorphism g": M -> M arbitrarily C' close to g' with the property that g' and g" are not topologically conjugate, at least by a homeomorphism pointwise close to the identity. (Thus g' is not a structurally stable diffeomorphism; see [10], [11] for elaboration of this). One can best study a small C' perturbation g' of g by using theorems of Perron [91 put into a general setting in the spirit of Anosov [2]. We expect to expand on this point in a future paper. In the meantime we argue as follows. First, for g' Cl-close enough to g, there is an invariant torus T' C M=1 T X S' corresponding to T X 0 and C'-close to it. For this see Kupka [4] or Moser [6]. Next, g' restricted to T' is topologically conjugate by a homeomorphism close to the identity to g: T X 0 -> T X 0; see [2] or [3]. Let the stable manifolds of g' of the countable dense set of periodic points qi' on T' be denoted by W28(qi')'. The tangent 2-plane to each point of W28(qi')' is close (arbitrarily, depending on the C' distance of g' to g) to the plane W28(qj) (which is independent of i, of course) on T X [- 1, 1]. 494 S. SMALE.

This is a consequence of Section 5 of [13]. Similarly, the W28(qi')' filled up T X [-1, 1 ] densely. The intersections of W28(qt')' near the z-axis with the x-axis will be a countable dense set >'. Denote by W1uthe 1-dimensional unstable manifold of the fixed point (0, 0, 2) of g and by W1u'that corresponding to the perturbation g'. Now for all g' (and g") sufficiently close to g, we can have precisely one of two cases as follows. Case 1: The first point on W1,', near the bump f (a), ordering by >' is on some W,8(qj')'. Case 2: It is not. The following two facts finish our picture of g'. 1. By letting g"= g'+ qch (see the definition of g), arbitrarily small q, we can suppose g" is in Case 2 if g' is of Case 1 and vice versa. 2. If g' and g" are in the opposite cases there is no homeomorphism h of M close to the identity such that g'h = hg". This can be seeni by iterating g' and g". From any diffeomorphism g of a compact n-manifold M onto itself, one can construct in a canonical way, an ordinary differential equation X on an (n + 1) manifold with a cross-section g: M--M (see [11]). In this way the various relevant properties of g correspond to those of X. Therefore, from what we have shown, the main theorem follows. The example in this paper surely reduces the importance of the notion of structural stability. One might be further discouraged from studying the global qualitative theory of higher dimensional, ordinary differential equations. We believe, however, that this study can be constructive and are preparing a paper in this direction based on an axiom, which we consider to be of central importance, axiom A, described as follows. Let X be a smooth vector field on a compact manifold M which generates a 1-parameter flow t: M- M. Relative to pt a point p C M is called wandering if there is a neighborhood U of p such that qt (U) n U= 0 for all t with I t I greater than some to. We will denote the non-wandering points by Qx = Q C M. Then Q is a closed invariant subset of Ml/. If II: E -> M is a vector space bundle over a topological space, a 1- parameter group of V-B maps is a 1-parameter group of homeomorphisms tt: E > E such that for each tCE, '/t is a V-B map (fiber preserving and linear on each fiber). For example if 4t: M-- M is a smooth 1-parameter flow on a differentiable manifold, the derivative ,t: TM -> TM defines a I- STRUCTURALLY STABLE SYSTEMS. 495 parameter group of the V-B maps of the tangent bundle, called the derived flow. A 1-parameter group of V-B maps tt: E - E over a compact space M is called contracting if for some Riemannian structure on H: E- 1M, the following two estimates hold:

1) j t(v) jj? a jjv e-ct , t O, vC E,

2) j(v)j ?b jjvje-ct t? O,vCE.

Here a, b, c are positive constants, and jjv 11is the norm of v in the metric on the fiber to which v belongs. Finally At is called expanding if iP-t is contracting. For any closed invariant set Q C M of a smooth differentiable flow Pt: M->M, a U-structure on Q (generalizing Anosov [2]) is defined as follows. The tangent bundle T(M) of 1M1restricted to Q, TQ(M), is a Whitney sum TQ(M) = DQu 0 DQSE) X (topologically) such that EQ1u Du? +X, EQ-S DQs + are invariant under the derived flow ot: T(M) -- T(M) of kt, bt: EQSe EQ. is contracting (as a 1-parameter group of V-B maps) and Pt: EQu > EQUis expanding. Here X is the line bundle defined by X, X the tangent vector field generating kt. For a smooth flow ot: M -- M1of a compact manifold then our axiom is:

Axiom A. The set of non-wandering points Q C M of qct has a U- structure.

REFERENCES.

[1] A. A. Andronov and L. S. Pontriagin, " Systimes Grossiers," Doklady Akademii Nauk, vol. 14 (1937), pp. 247-251. [2] D. V. Anosov, "Roughness of geodesic flows on compact Riemannian manifolds of negative curvature," Soviet Mathematics, vol. 3 (1962), pp. 1068-1070. [3] V. I. Arnold and Ja. G. Sinai, " Small perturbations of the automorphisms of the torus," Soviet Mathematics, vol. 3 (1962), pp. 783-787. See also correction of same, Soviet Mathematics, vol. 4 (1963) page preceding 561. [4] I. Kupka, " Stabilite des varietes invariantes d'un champ de vecteurs pour les petites perturbations," Comptes Rendus de l'Academie des Sciences de Paris, vol. 258 (1964), pp. 4197-4200. [5] S. Lefschetz, Differential Equations: Geometric Theory, New York, 1957. 496 S. SMALE.

[6] 3. Moser, On invariant manifolds of vector fields and symmetric partial differential equations, in Differential Analysis, Bombay, 1964. [7] M. Peixoto, " On structural stability," Annals of Mathematics, vol. 69 (1959), pp. 199-122. [8] ," Structural stability on 2-dimensional manifolds," Topology, vol. 2 (1962), pp. 101-121. [9] Perron, "Die Stabilitatsfrage bei Differentialgleichungen," Mathematische Zeit- schrift, vol. 32 (1930), pp. 703-728. [10] S. Smale, Dynamical systems and the topological conjugacy problem for diffeomorphisms, Proceedings of the International Congress of Mathematicians, Stockholm, 1962. [11] , " Stable manifolds for differential equations and diffeomorphisms," Annali della Scuola Normale Superiore di Pisa, Series III, vol. XVII (1963), pp. 97-116. [12] , A structurally stable differentiable homeomorphism with an infinite number of periodic points, Report on the Symposium on non-linear oscilla- tions, Kiev Mathematical Institute, 1961. [13] , Diffeomorphisms with many periodic points, M. Morse Symposium, Dif- ferential and Combinatorial Topology, Princeton, 1965.