TRANSACTIONS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 203, 1975 THE AUTOMORPHISMGROUP OF A COMPACTGROUP ACTION BY W. D. CURTIS ABSTRACT. This paper contains results on the structure of the group, DiffG(M), of equivariant C-diffeomorphisms of a free action of the compact Lie group G on M. DiffJjíAÍ) is shown to be a locally trivial principal bundle over a submanifold of T>\tf(X), X the orbit manifold. The structural group of this bundle is lf(G, M), the set of equivariant C'-diffeomorphisms which induce the identity on X. E?(G, M) is shown to be a submanifold of Diff^Af) and in fact a Banach Lie group (r < ~). 0. Introduction. This paper studies the group of equivariant diffeomorphisms of a smooth action of a compact Lie group G on a compact manifold M. Spe- cifically most of the paper deals with the case when G acts freely on M. In this case there is an orbit manifold X, and an equivariant C-diffeomorphism / of M induces a C-diffeomorphism / of X. This defines a homomorphism P: DiffG(Af)—► Diff(X) (DiffG(A/) is the group of equivariant diffeomorphisms on M of class C, 1 < r < °°). We obtain some results about the structure of P. We show P admits smooth local cross-sections (Theorem 3.5). The kernel of P is the group of C-equivariant diffeomorphisms of M which induce the identity on X. This group is the structural group of the locally trivial principal bundle determined by P. We show ker F is a smooth submanifold of Diff(A/) and, that with respect to the induced differential structure, ker P is a Banach Lie group (Theorem 4.2). Recall Diff(M) is not a Banach Lie group as composition is not C1 (r < °°). The main technique introduced is the construction in §2 of G-lifts of sprays on X. This allows a precise connection between the manifold of maps differential structures on DiffG(7W) and Diff(X). 1. Preliminaries. M will always denote a compact, connected, C°°-mani- fold, G a compact Lie group. Assume G acts on M (on the left). We denote by X the orbit space and we let n: M —> X be the orbit projection. If G acts freely and differentiably (C°°) then X has a natural C°°-structure such that tt Presented to the Society, January 26, 1973 under the title The automorphism group of a compact Lie group action; received by the editors March 22, 1973. AMS (A/OS) subject classifications (1970). Primary 58D05, 22E65; Secondary 55F10. Key words and phrases. Diffeomorphism group, equivariant diffeomorphism, Banach Lie group, principal bundle, spray, free action. Copyright © 1975, American Mathematical Society License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use45 46 W. D. CURTIS is C°° and locally trivial (since G is compact). If g G G, w G M we write gw to denote the result of letting g act on w. We shall also write g to denote the diffeomorphism w —* gw (henceforth all manifolds, actions, etc. are assumed C°°). An action of G on M induces an action of G on TM, the tangent bundle of M If g G G we write Tg(v) for the result of acting on v G TM by g. The resulting diffeomorphism of TM is Tg: TM —►TM and is just the tangent of g: M —►M. Similarly G acts on T2M = T(TM) by "double tangents", T2g = T(Tg): T2M-+ T2M being the diffeomorphism corresponding to the group element g. Definition 1.1. Let G act on M. An automorphism of class C is a diffeomorphism /: M —►M of class C which is G-equivariant. We denote by DiffG(iW) the group of all C-automorphisms of the action. (Here l<r<°°.) Let Diffr(M) be the group of all C-diffeomorphisms of M onto M. If G acts on M then DiffG(Af) is clearly a subgroup of Diff(M). The differential structure on Diffr(A0: We review the standard manifold of maps construction as applied to make Diff(Af) into a C°° -manifold. We recall that Diff(M) is locally Banach if 1 < r < °° while being locally Fréchet in the case r = °°. Also in case r = °° the multiplication and inversion in Diffr(M) are C°° so Diff°°(M) is a Fréchet Lie group [5]. Let ¿f: TM —* T2M be a C°-spray. There is an open neighborhood 0 of the 0-section in TM and an open neighborhood U of the "diagonal" in M x M and a C°°-diffeomorphism Exp: 0 —* U defined by Exp(u) = (t(v), exp(u)) where t: TM —*■M is projection and exp is the exponential of the spray |. We use Exp to construct "natural charts" around each / G Diffr(M). We define a neighborhood Nf of / by Nf = {h G Difr(M)\(f(x), h(x)) GU for all xGM). Letting rr(f*TM) denote the space of all C-sections of the induced bundle f*TM, with the C-topology, we define af: Nf —►V(f*TM) by a/h)(x) = (x,Exp~l(f(x),h(x))). af maps Nf bijectively onto an open set Tr(f*TM) and the collection of all charts (Nf, af) for /G Diff(M) is a C°-atlas. This C°-structure is independent of the choice of spray. For further discussion of this manifold of maps construction, we refer to [1], [3], [5]. For use later we now consider some properties of G-invariant sprays. Definition 1.2. If G acts on M we say a spray %: TM —+ T2M is G-invariant if, for all g CG, we have T2g ° %= %° Tg. The following proposition is easily proved. Proposition 1.3. // % is a G-invariant spray, then the domain of exp^ is an invariant subset of TM and exp^ is G-equivariant. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use THE AUTOMORPHISM GROUP OF A COMPACT GROUP ACTION 47 Using notation as before we have Lemma 1.4. // £ is a G-invariant spray then 0 may be chosen to be G-invariant. Exp: 0 —*■M x M is then G-equivariant, where ' G acts on M x M by g(xvx2) = (gxv gx2). It then follows that f/=Exp(0) is an invariant set. Proof. Let Qx be a neighborhood of the 0-section such that Exp isa diffeomorphism on 0¡. If we choose a G-invariant Riemannian metric on M, then for sufficiently small e > 0 we will have We = {v G TM\Hull < e} C Oj (since M is compact). But W£ is G-invariant since we have used an invariant metric. If g G G, v G 0 = We we have Exp(7g • v) = (r(Tg ■ v), exp(Tg ■ v)) = (%t(v), g exp(u)) = g Exp(u). This proves the lemma. 77ie group DiffG(M): If G acts freely on M then there is a homomor- phism P: DiffG(Af) —►Diff(X). The main results of this paper are concerned with the structure of P. P is defined by P(f) = /j where /j makes the follow- ing diagram commute. /. A preliminary result is Proposition 1.5. DiffG(A/) is a submanifold of Diff(M) for any action of G on M (free or not). Proof. Let % be an invariant spray. Such always exist, e.g. take % to be the geodesic spray of an invariant Riemann metric. We show that the natural chart at the identity map is a submanifold chart. We have a: N -* rr(TM), a(/)(x) = Exp" »(x, fix)). Let TrG(TM)C ^(TM) consist of all C-vectorfields f on M such that Tg ° f = f ° g for all g G G (i.e., all invariant vectorfields). Using the fact that by Lemma 1.4 Exp is equivariant, one immediately sees that afTVn DiffG(M)) = a(N) n ^(TM). This proves the proposition. Assumption. Henceforth we consider only free actions of G on M. In order to investigate the homomorphism P defined above we shall, in the next section, construct a special type of spray on M. This spray will allow us to relate the differential structure on DiffG(A/) to that on Diff(X). 2. Construction of G-lifts of sprays. Suppose G acts freely on M, n: M —*■X orbit projection. If 17 is a spray on X then a spray on M, say £, License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 48 W. D. CURTIS is called a lift of r? if the following commutes. TM—*-+ T2M Tn T2n TX ■+rx If, in addition, % is G-invariant we will say % is a G-lift of 77. We shall prove Theorem 2.1. If 77 is any spray on X then there exists a spray % on M which is a G-lift of 77. The proof of Theorem 2.1 will be based on several lemmas. Let {Ui}i (1 in some index set) be an open cover for X. Let Mt = W_I(0/) for each 1. {M¡)¡ is an open cover of M and {TM¡ = t~1(M¡)}¡ is an open cover of TM. Lemma 2.2. There is a C°°-partition of unity {</?,.},.,subordinate to {TM¡}¡ such that each <çt is G-invariant, i.e., <pi°Tg = ipi for all gGG. Proof. Choose a C-partition of unity, {</'/}<subordinate to {U¡}¡. Let <fi¡= ip¡ o n » t. {<p¡}¡is easily seen to have the desired properties. The next lemma is general in nature.