Trudy Moskov. Matem. Obw. Trans. Moscow Math. Soc. Tom 66 (2005) 2005, Pages 169–241 S 0077-1554(05)00149-4 Article electronically published on November 9, 2005 TYPES OF INTEGRABILITY ON A SUBMANIFOLD AND GENERALIZATIONS OF GORDON’S THEOREM N. N. NEKHOROSHEV Abstract. At the beginning of the paper the concept of Liouville integrability is analysed for systems of general form, that is, ones that are not necessarily Hamil- tonian. On this simple basis Hamiltonian systems are studied that are integrable only on submanifolds N of the phase space, which is the main subject of the paper. The study is carried out in terms of k-dimensional foliations and fibrations defined on N by the Hamiltonian vector fields corresponding to k integrals in involution. These integrals are said to be central and may include the Hamiltonian function of the system. The parallel language of sets of functions is also used, in particular, sets of functions whose common level surfaces are the fibres of fibrations. Relations between different types of integrability on submanifolds of the phase space are established. The main result of the paper is a generalization of Gordon’s theorem stating that in a Hamiltonian system all of whose trajectories are closed the period of the solutions depends only on the value of the Hamiltonian. Our generalization asserts that in the case of the strongest “Hamiltonian” integrability the frequencies of a conditionally periodic motion on the invariant isotropic tori that form a fibration of an integrability submanifold depend only on the values of the central integrals. Under essentially weaker assumptions on the fibration of the submanifold into such tori it is proved that the circular action functions also have the same property. In addition, certain general recipes for finding the integrability submanifolds are given. 0. Introduction The main object of investigation in this paper is autonomous Hamiltonian systems that are not necessarily integrable. Such a system is given by its Hamiltonian function H : M → R defined on a symplectic manifold M = M 2n, that is, a manifold equipped with some symplectic structure ω2,where2n =dimM 2n. Integrable Hamiltonian systems have a relatively simple structure and are fairly well studied, at least in the regular situation. (In the case of “nonregular” integrability there are many interesting questions; see, for example, [9].) But most systems are not integrable on the entire phase space M even in the nonregular sense. What is the structure of the phase flow of nonintegrable systems? It is known that many such systems are integrable, but only on some invariant submanifolds, rather than on the whole of M. First we will look at the regular toric integrability of the system on a submanifold b N ⊆ M. By this we mean that the k-dimensional invariant tori Λ form a locally trivial b fibration of this submanifold, so that N = b∈B Λ , where the parameter b parametrizing 2000 Mathematics Subject Classification. Primary 37J05, 70H12; Secondary 37J15, 37J35, 37J45. Key words and phrases. Integrable, Hamiltonian, Gordon’s theorem, integrability submanifold, condi- tionally periodic motion, invariant tori, vector field, integrals in involution, symplectic structure, circular action functions, frequency, trajectory, isotropic tori. This paper was written with partial support of the INTAS grant no. 00-221 and the research was partially carried out during the author’s stay at the Littoral University, Laboratory UMR 8101 of CNRS, Dunkerque, France, and at the Milan University, Italy. c 2005 American Mathematical Society 169 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 170 N. N. NEKHOROSHEV these tori runs over some manifold B. The invariance means that the tori Λb are invariant { t ∈ } t b b under the action of the local phase flow gH , t R of the system, that is, gH Λ =Λ for any time t. We also assume that the motion of the points of the phase space under the action of this flow is conditionally periodic on the tori Λb. Studying the integrability submanifolds appears to be important for the following reasons. It often turns out that it is such submanifolds N ⊂ M, as well as their neigh- bourhoods in the whole phase space M of the system, that are of primary interest for applications. In addition, in a neighbourhood of an integrability submanifold the phase flow of the system has the simplest structure. We think that studying the integrability submanifolds is the easiest, but also the most important and immediate step in the study of nonintegrable Hamiltonian systems. In the present paper we are interested only in the behaviour of the system on the submanifold N itself. There is an extensive literature devoted to finding families of integrable solutions. For example, many examples of this kind of particular solutions to the equations of the dynamics of a rigid body are described in the book [24], which also contains a large list of references related to this problem. Numerous families of integrable solutions were found for the equations of celestial mechanics and many other quite diverse differential equations of importance in applications. In our context the trajectories of all these solutions form integrability submanifolds. The purpose of our paper is to study general approaches to FINDING the integrability submanifolds, clarifying the CAUSES of existence of such submanifolds, and investigating the QUALITY OF INTEGRABILITY of systems on them, especially, for Hamiltonian systems, and toric integrability. It is here that when we have the highest-quality integra- bility the generalization of Gordon’s theorem for frequencies holds. Note that finding the toric integrability submanifolds interlocks with finding the families of periodic solutions of dynamical systems the unions of whose trajectories can be regarded as a special case of integrability submanifolds; see, in particular, [7, 6]. In fact, the quality of integrability on a given submanifold N ⊆ M, as it is understood here, depends on how fully the Hamiltonian character of the system on N is used. One may consider that this character is taken into account most fully in the case of “complete integrability”, that is, the integrability of a Hamiltonian system on the entire phase space M, which is usually studied. This type of integrability on the whole of M has been studied from general differential-geometric viewpoints in [1, 5, 3] and many other papers. The case where an integrability submanifold N ⊆ M of the symplectic manifold M is symplectic is easily reduced to the case of integrability on the entire symplectic phase space. The fact is that the restriction of a Hamiltonian system to an invariant symplectic submanifold is also a Hamiltonian system; see Lemma 3.1.1. So if this system with p degrees of freedom, defined on N = N 2p, is completely Hamiltonian integrable, its phase 2p 2 2 2 space (N , ω˜ ), whereω ˜ := ω |N , is fibred into the k-dimensional invariant isotropic surfaces Λ; here k ≤ p ≤ n. If these surfaces are compact sets, then they are diffeomorphic to tori, on which the motion of the system is conditionally periodic. Furthermore, in the case where N is sym- plectic one can construct normalizing canonical “coordinates” (I,p, ϕ mod 2π,q)onN that are generalizations of the action-angle variables (I, ϕ mod 2π). In these coordinates the Hamiltonian H˜ := H|N of the system with Hamiltonian H restricted to N has the form H˜ = H˜ (I) (see Proposition 5.1.1); more details about these coordinates can be found in [5]. Hence the frequencies of the conditionally periodic motion on these tori, ωi =˙ϕi = ∂H/∂I˜ i, i =1,...,k, depend only on the action variables I. This assertion is equivalent to the generalization of Gordon’s theorem for frequencies. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use TYPES OF INTEGRABILITY ON A SUBMANIFOLD 171 Two simple examples of such a situation are given in § 1.Namely,weproducetwo systems, both in all probability nonintegrable, that are integrable on certain symplectic submanifolds N ⊂ M. In addition, in § 1 we also illustrate Gordon’s theorem itself by the example of the problem of two and many bodies. (The precise statement of the original version of this theorem is given at the beginning of § 5.3.) Thus, the case where an integrability submanifold N ⊂ M is symplectic is relatively simple. But the case where N ⊂ M is a submanifold of general form, for example, of odd dimension, is more complicated. The question arises as to how we can take into account the Hamiltonian characteristics most fully in this case? And under what conditions do the generalizations of Gordon’s theorem remain valid? In order to answer these questions we must find the basic types of integrability that a Hamiltonian system can have on a submanifold. In turn, in order to understand this we need to clearly understand the much more simple concept of Liouville integrability for systems of general form, that is, systems that are not necessarily Hamiltonian. This concept is considered in § 2, which is entirely devoted to systems of general form, which are not considered further in the paper. For such a system with phase space N to be Liouville integrable there must be k vector fields (a1,...,ak)=A on N that are linearly independent at each point, that are pairwise commutative, and that commute with the vector field V of the given system. (One of the fields in the set A may coincide with the given field V , but this is not necessary.) In addition, we require that there exist r functionally independent integrals common to all these k + 1 vector fields, and that k + r =dimN.