GAUGED FLOER HOMOLOGY for HAMILTONIAN ISOTOPIES I: DEFINITION of the FLOER HOMOLOGY GROUPS Contents 1. Introduction 1 2. Basic S

Home , Andreas Floer, Dietmar Salamon, Morse homology

GAUGED FLOER HOMOLOGY FOR HAMILTONIAN ISOTOPIES I: DEFINITION OF THE FLOER HOMOLOGY GROUPS

GUANGBO XU

Abstract. We construct the vortex Floer homology group VHF (M, µ; H) for an aspherical Hamil-

tonian G-manifold (M, ω) with moment map µ and a class of G-invariant Hamiltonian loop Ht, following the proposal of [3]. This is a substitute for the ordinary Hamiltonian Floer homology of the symplectic quotient of M. We achieve the transversality of the moduli space by the classical perturbation argument instead of the virtual technique, so the homology can be deﬁned over Z or Z2.

Contents 1. Introduction 1 2. Basic setup and outline of the construction7 3. Asymptotic behavior of the connecting orbits 14 4. Fredholm theory 21 5. Compactness of the moduli space 28 6. Floer homology 31 Appendix A. Transversality by perturbing the almost complex structure 37 References 46

1. Introduction 1.1. Background. Floer homology, introduced by Andreas Floer (see [8], [9]), has been a great triumph of J-holomorphic curve technique invented by Gromov [17] in many areas of mathematics. Hamiltonian Floer homology gives new invariants of symplectic manifolds and its Lagrangian submanifolds and has been the most important approach towards the solution to the celebrated Arnold conjecture initiated in the theory of Hamiltonian dynamics; the Lagrangian intersection Floer homology is the basic language in defining the Fukaya category of a symplectic manifold and stating Kontsevich’s homological mirror symmetry conjecture; several Floer-type homology theory, including the instanton Floer homology ([7], [4]), Heegaard-Floer theory ([29]), Seiberg- Witten Floer homology ([22]), ECH theory ([20], [21]), has become tools of understanding lower dimensional topology. All these different types of Floer theory, are all certain infinite dimensional Morse theory, whose constructions essentially apply Witten’s point of view ([34]). Basically, if f : X → R is certain

Date: December 24, 2013. 1 2 GUANGBO XU smooth functional on manifold X (which could be inﬁnite dimensional), then with an appropriate choice of metric on X, we can study the equation of negative gradient ﬂow of f, of the form x0(t) + ∇f(x(t)) = 0, t ∈ (−∞, +∞). (1.1)

If some natural energy functional defined for maps from R to X is finite for a solution to the above equation, then x(t) will converges to a critical point of f. Assuming that all critical points of f is nondegenerate, then usually we can define a Morse-type index (or relative indices) λf : Critf → Z. Then for a given pair of critical points a−, a+ ∈ Critf, the moduli space of solutions to the negative gradient flow equation which are asymptotic to a± as t → ±∞, denoted by M(a−, a+), has dimension equal to λf (a−) − λf (a+), if f and the metric are perturbed generically. If λf (a−) − λf (a+) = 1, because of the translation invariance of (1.1), we expect to have only finitely many geometrically different solutions connecting a− and a+. In many cases (which we call the oriented case), we can also associate a sign to each such solutions. On the other hand, we define a chain complex over Z2 (and over Z in the oriented case), spanned by critical points of f and graded by the index λf ; the boundary operator ∂ is defined by the (signed) counting of geometrically different trajectories of solutions to (1.1) connecting two critical points with adjacent indices. We expect a nontrivial fact that ∂ ◦ ∂ = 0. So a homology group is derived.

1.2. Hamiltonian Floer homology and the transversality issue. In Hamiltonian Floer theory, we have a compact symplectic manifold (X, ω) and a time-dependent Hamiltonian Ht ∈ ∞ C (X), t ∈ [0, 1]. We can define an action functional AH on a covering space LXg of the contractible loop space of X. The space LXg consists of pairs (x, w) where x : S1 → X is a contractible loop and w : D → X with w|∂D = x; the action functional is defined as Z Z ∗ AH (x, w) = − w ω − Ht(x(t))dt. (1.2) D S1 The Hamiltonian Floer homology is formally the Morse homology of the pair LX,g AH . 1 0 The critical points are pairs (x, w) where x : S → X satisfying x (t) = XHt (x(t)), where XHt is the Hamiltonian vector field associated to Ht; these loops are 1-periodic orbits of the Hamiltonian 1 isotopy generated by Ht. Then, choosing a smooth S -family of ω-compatible almost complex 2 structures Jt on X which induces an L -metric on the loop space of X,(1.1) is written as the Floer 1 equation for a map u from the infinite cylinder Θ = R × S to X, as ∂u ∂u + J − X (u) = 0. (1.3) ∂s t ∂t Ht Here (s, t) is the standard coordinates on Θ. This is a perturbed Cauchy-Riemann equation, so Gromov’s theory of pseudoholomorphic curves is adopted in Floer’s theory. There is always an issue of perturbing the equation in order to make the moduli spaces transverse, so that the ambiguity of counting of solutions doesn’t affect the resulting homology. Floer originally defined the Floer homology in the monotone case, which was soon extended by Hofer-Salamon ([19]) and Ono ([28]) to the semi-positive case. Finally by applying the “virtual technique”, Floer homology is defined for general compact symplectic manifold by Fukaya-Ono ([15]) and Liu-Tian ([23]). GAUGED FLOER HOMOLOGY 3

1.3. Hamiltonian Floer theory in gauged σ-model. In this paper, we consider a new type of Floer homology theory proposed in [3] and motivated from Dostoglou-Salamon’s study of Atiyah- Floer conjecture (see [6]). The main analytical object is the symplectic vortex equation, which was also independently studied initially in [3] and by Ignasi Mundet in [25]. The symplectic vortex equation is a natural elliptic system appearing in the physics theory “2-dimensional gauged σ-model”. Its basic setup contains the following ingredients:

(1) The target space is a triple (M, ω, µ), where (M, ω) is a symplectic manifold with a Hamil- tonian G-action, and µ is a moment map of the action. We also choose a G-invariant, ω-compatible almost complex structure J on M. (2) The domain is a triple (P, Σ, Ω), where Σ is a Riemann surface, P → Σ is a smooth G- bundle, and Ω is an area form on Σ.

(3) The “ﬁelds” are pairs (A, u), where A is a smooth G-connection on P , and u :Σ → P ×G M is a smooth section of the associated bundle.

Then we can write the system of equation on (A, u):

( ∂ u = 0; A (1.4) ∗FA + µ(u) = 0.

Here ∂Au is the (0, 1)-part of the covariant derivative of u with respect to A; FA is the curvature 2-form of A; ∗ is the Hodge star operator associated to the conformal metric on Σ with area form Ω; µ(u) is the composition of µ with u, which, after choosing a biinvariant metric on g, is identiﬁed with a section of adP → Σ. This equation contains a symmetry under gauge transformations on P . Moreover, its solutions are minimizers of the Yang-Mills-Higgs functional:

1 2 2 2 YMH(A, u) := kd uk 2 + kF k 2 + kµ(u)k 2 (1.5) 2 A L A L L which generalizes the Yang-Mills functional in gauge theory and the Dirichlet energy in harmonic map theory. Now, similar to Hamiltonian Floer theory, consider the following action functional on a covering 1 1 space of the space of contractible loops in M × g. Let H : M × S → R be an S -family of G- 1 invariant Hamiltonians; for any contractible loop xe := (x, f): S → M × g with a homotopy class 1 of extensions of x : S → M, represented by w : D → M, the action functional (given ﬁrst in [3]) is

Z Z ∗ AeH (x, f, w) := − w ω + (µ(x(t)) · f(t) − Ht(x(t))) dt. (1.6) D S1

The critical loops of AeH corresponds to periodic orbits of the induced Hamiltonian on the symplectic −1 quotient M := µ (0)/G. The equation of negative gradient ﬂows of AeH , is just the symplectic vortex equation on the trivial bundle G × Θ, with the standard area form Ω = ds ∧ dt, and the connection A is in temporal gauge (i.e., A has no ds component). If choosing an S1-family of

G-invariant, ω-compatible almost complex structures Jt, then the equation is written as a system 4 GUANGBO XU of (u, Ψ) : Θ → M × g:

 ∂u ∂u  + J + X (u) − Y = 0;  t Ψ Ht ∂s ∂t (1.7) ∂Ψ  + µ(u) = 0.  ∂s

Solutions with finite energy are asymptotic to loops in CritAeH . Then the moduli space of such trajectories, especially those zero-dimensional ones, gives the definition of the boundary operator in the Floer chain complex, and hence the Floer homology group. We call these homology theory the vortex Floer homology. We have to use certain Novikov ring Λ, which will be defined in Section 2, as the coefficient ring, and the vortex Floer homology will be denoted by VHF (M, µ; H,J; Λ). The main part of this paper is devoted to the analysis about (1.7) and its moduli space, in order to define VHF (M, µ; H,J; Λ).

1.4. Lagrange multipliers. The action functional (1.6) seems to be already complicated, not to mention its gradient flow equation (1.7). However, the action functional (1.6) is just a Lagrange multiplier of the action functional (1.2). Indeed there is a much simpler situation in the case of the Morse theory of a finite-dimensional Lagrange multiplier function, which is worth mentioning in this introduction as a model. Suppose X is a Riemannian manifold and µ : X → R is a smooth function, with 0 a regular −1 value. Then consider a function f : X → R whose restriction to X = µ (0) is Morse. Then critical points of f|X are the same as critical points of the Lagrange multiplier F : X × R → R defined by F (x, η) = f(x) + ηµ(x), and the Morse index as a critical point of f|X is one less than the index as a critical point of F . Then instead of considering the Morse-Smale-Witten complex of f|X , we can consider that of F . In generic situation, these two chain complexes have the same homology (with a grading shifting), and a concrete correspondence can be constructed through the “adiabatic limit” (for details, see [33]). Indeed, the vortex Floer homology proposed by Cieliebak-Gaio-Salamon and studied in this paper is an infinite-dimensional and equivariant generalization of this Lagrange multiplier technique. Therefore, the vortex Floer homology is expected to coincide with the ordinary Hamiltonian Floer homology of the symplectic quotient (the proof of this correspondence will be treated in separate work).

1.5. Advantage in achieving transversality. It seems that by considering the complicatd equation (1.7) and the moduli spaces we can only recover what we have known of the Hamiltonian Floer theory of the symplectic quotient. But the trade-off is that the most crucial and sophisticated step–transversality of the moduli space–can be achieved more easily. The advantage of lifting to gauged σ-model is because, in many cases, M has simpler topology than M. So the issue caused by spheres with negative Chern numbers is ruled out by topological reason. This phenomenon allows us to achieve transversality of the moduli space by using the traditional “concrete perturbation” to the equation. Moreover, when using virtual technique, the Floer homology group of the symplectic quotient can only be defined over Q but here it can be defined over Z or Z2. GAUGED FLOER HOMOLOGY 5

1.6. Computation of the Floer homology group and adiabatic limits. The ordinary Hamil- tonian Floer homology HF (M,H) of a compact symplectic manifold can be shown to be canonically independent of the Hamiltonian H and to be isomorphic to the singular homology of M. This correspondence plays significant role in proving the Arnold conjecture. To prove this isomorphism, basically two methods have been used. One is to use a time-independent Morse function as the Hamiltonian, and try to prove that when the function is very small in C2-norm, there is no “quantum contribution” when defining the boundary operator in the Floer chain complex; this was also Floer’s original argument. Another is via the Piunikhin-Salamon-Schwarz (PSS) construction, introduced in [30]. For the case of the vortex Floer homology, it has been well-expected to be isomorphic to the singular homology of the symplectic quotient M. To prove this isomorphism we can try the similar methods as for the ordinary Hamiltonian Floer homology (which we will discuss in Section 6.3), as well as the adiabatic limit method, which we discuss here. Indeed, for any λ > 0, we consider a variation of (1.7)  ∂u ∂u  + J + X (u) − Y = 0;  t Ψ Ht ∂s ∂t (1.8) ∂Ψ  + λ2µ(u) = 0.  ∂s 1 This can be viewed as the symplectic vortex equation over the cylinder R × S with area form replaced by λ2dsdt. The moduli space of solutions to the above equation also defines a Floer homology group, and by continuation method we can show that this homology is (canonically) independent of λ. Then we would like to let λ approach to ∞. By a simple energy estimate, solutions of (1.8) will “sink” into the symplectic quotient M and become Floer trajectors of the induced pair (H, J); at isolated points there will be energy blow up, and certain “affine vortices” will appear, which are finite energy solutions to the symplectic vortex equation over the complex plane C. In the Gromov- Witten setting, the work of Gaio-Salamon [16] shows that (in special cases), the Hamiltonian- Gromov-Witten invariants with low degree insertions coincide with the Gromov-Witten invariants ∗ ∗ of the symplectic quotient, via the Kirwan map κ : HG(M) → H (M). The high degree part shall be corrected, by the contribution from the affine vortices. This leads to the definition of the “quantum Kirwan map” (see [38], [35]). In the case of vortex Floer homology, as long as we can carefully analyze the contribution of affine vortices (maybe with similar restriction on M as in [16]), we could prove that VHF (M, µ; H) is isomorphic to HF (M; H), with appropriate changes of coefficients. It is an interesting topic to consider the reversed limit λ → 0, and it actually motivated the work of the author with S. Schecter [33], where they considered the nonequivariant, finite dimensional Morse homology. In [33] it was shown that, the Morse-Smale trajectories, as λ → 0, will converge to certain “fast-slow” trajectories, and the counting of such trajectories defines a new chain complex, which also computes the same homology.

1.7. Gauged Floer theory for Lagrangian intersections. In Frauenfelder’s thesis and [12], he used the symplectic vortex equation on the strip R × [0, 1] to deﬁne the “moment Floer homology” 6 GUANGBO XU for certain types of pairs of Lagrangians (L0,L1) in M. The Lagrangians are not G-invariant −1 in general, but their intersections with µ (0) reduce to a pair of Lagrangians (L0, L1) in the symplectic quotient M. Then by the calculation in the Morse-Bott case, he managed to prove the Arnold-Givental conjecture with certain topological assumption on M. Woodward also deﬁned a version of gauged Floer theory in [36], where he considered a pair of

Lagrangians L0, L1 in the symplectic quotient M. They lift to a pair of G-invariant Lagrangians −1 L0,L1 ⊂ µ (0) ⊂ M. Then his equation for connecting orbits is the naive limit of the symplectic vortex equation on the strip R × [0, 1], by setting the area form to be zero. Since the strip is contractible, the equation is just the J-holomorphic equation on the strip, and two solutions are regarded equivalent if they differ by a constant gauge transformation. Then he applied this Floer theory to the fibres of the toric moment map for any toric orbifold and showed the relation between the nondisplacibility of toric fibres and the Hori-Vafa potential, which reproduces and extends the results of Fukaya et. al. [13][14]. Both of the above take advantage of the simpler topology of M than the symplectic quotient, as we mentioned above, to avoid certain virtual technique. Further work are expected to relate the Hamiltonian gauged Floer theory we studied here and the Lagrangian versions, for example, by constructing the so-called “open-close map”.

1.8. Organization and conventions of this paper. In Section2 we give the basic setup, including the action functional, the definition of the Floer chain complex and the equation of connecting orbits. In Section3 we proved that each finite energy solution is asymptotic to critical loops of the action functional. In Section4 we study the Fredholm theory of the equation of connecting orbits (modulo gauge transformations); we show that the linearized operator is a Fredholm operator whose index is equal to the difference of Conley-Zehnder indices of the two ends of the connecting orbit. In Section5 we prove that our moduli space is compact up to breaking, if assuming the nonexistence of nontrivial holomorphic spheres. In Section6 we summarize the previous constructions and give the definition of the vortex Floer homology (where we postpone the proof of transversality). We also prove the invariance of the homology group by using continuation method. In the final Section we give some discussions on our further work along this line. In the appendices we provide detailed proof of several technical theorems. Most importantly, we showed that by using concrete perturbation of the almost complex structure, we can achieve transversality of the moduli space, which allows us to avoid the more sophisticated virtual technique. 1 We use Θ to denote the infinite cylinder R×S , with the axial coordinate s and angular coordinate 1 1 t. We denote Θ+ = [0, +∞) × S and Θ− = (−∞, 0] × S . G is a connected compact Lie group, with Lie algebra g. Any G-bundle over Θ is trivial, and we just consider the trivial bundle P = G × Θ. Any connection A can be written as a g-valued 1-form on Θ. We always use Φ to denote its ds component and Ψ to denote its dt component. ∗ ∗ ∗ −1 ∗ There is a small > 0 such that for the -ball g ⊂ g centered at the origin of g , U := µ (g ) −1 ∗ −1 can be identified with µ (0) × g . We denote by πµ : U → µ (0) the projection on the the first −1 component, and by πµ : U → M the composition with the projection µ (0) → M. GAUGED FLOER HOMOLOGY 7

1.9. Acknowledgments. The author would like to thank his PhD advisor Gang Tian for introduc- ing him to this ﬁeld and the support and encouragement. He would like to thank Urs Frauenfelder, Kenji Fukaya, Chris Woodward, and Weiwei Wu for many helpful discussions and encouragement.

2. Basic setup and outline of the construction Let (M, ω) be a symplectic manifold. We assume that it is aspherical, i.e., for any smooth map Z f : S2 → M, f ∗ω = 0. This implies that for any ω-compatible almost complex structure J on S2 M, there is no nonconstant J-holomorphic spheres. Let G be a connected compact Lie group which acts on M smoothly. The inﬁnitesimal action g 3

ξ 7→ Xξ ∈ Γ(TM) is an anti-homomorphism of Lie algebra. We assume the action is Hamiltonian, which means that there exists a smooth function µ : M → g∗ satisfying

−1 µ(gx) = µ(x) ◦ Adg , ∀ξ ∈ g, d (µ · ξ) = ιXξ ω. (2.1) ∞ 1 Suppose we have a G-invariant, time-dependent Hamiltonian H ∈ Cc M × S with compact 1 support. For each t ∈ S , the associated Hamiltonian vector ﬁeld YHt ∈ Γ(TM) is determined by 1 ω(YHt , ·) = dHt ∈ Ω (M). (2.2)

The ﬂow of YHt is a one-parameter family of diﬀeomorphisms dφH (x) φH : M → M, t = Y φH (x) (2.3) t dt Ht t which we call a Hamiltonian path.

To achieve transversality, we put the following restriction on Ht. It will be used only in the appendix.

1 Hypothesis 2.1. There exists a nonempty interval I ⊂ S such that Ht ≡ 0 for t ∈ I.

On the other hand, we need to put several assumptions to the given structures, which are still general enough to include the most important cases (e.g., toric manifolds as symplectic quotients of Euclidean spaces).

Hypothesis 2.2. We assume that µ : M → g∗ is proper, 0 ∈ g∗ is a regular value of µ and the G-action restricted to µ−1(0) is free.

With this hypothesis, µ−1(0) is a smooth submanifold of M and the symplectic quotient M := µ−1(0)/G is a symplectic manifold, which has a canonically induced symplectic form ω. Also, the

Hamiltonian function Ht descends to a time-dependent Hamiltonian

Ht : M → R (2.4) −1 by the G-invariance of Ht. It is easy to check that YHt is tangent to µ (0) and the projection µ−1(0) → M pushes Y forward to Y . Then we assume Ht Ht

Hypothesis 2.3. The induced Hamiltonian Ht : M → R is nondegenerate in the usual sense. Finally, in the case when M is noncompact, we need the convexity assumption (cf. [2, Deﬁnition 2.6]). 8 GUANGBO XU

Hypothesis 2.4. There exists a pair (f, J), where f : M → [0, +∞) is a G-invariant and proper function, and J is a G-invariant, ω-compatible almost complex structure on M, such that there exists a constant c0 > 0 with

f(x) ≥ c0 =⇒ h∇ξ∇f(x), ξi + h∇Jξ∇f(x), Jξi ≥ 0, df(x) · JXµ(x) ≥ 0, ∀ξ ∈ TxM. (2.5)

In this paper, to achieve transversality, we need to perturb J near µ−1(0) (see the appendix). The above condition is only about the behavior “near inﬁnity”, so such perturbations don’t break the hypothesis.

2.1. Equivariant topology.

2.1.1. Equivariant spherical classes. Recall that the Borel construction of M acted by G is MG := EG ×G M, where EG → BG is a universal G-bundle over the classifying space BG. Then the equivariant (co)homology of M is deﬁned to be the ordinary (co)homology of MG, denoted by G ∗ H∗ (M) for homology and HG(M) for cohomology. On the other hand, for any smooth manifold N, we denote by S2(N) to be the image of the Hurwitz map π2(N) → H2(N; Z), and classes in S2(N) are called spherical classes. We deﬁne the G equivariant spherical homology of M to be S2 (M) := S2(MG). G Geometrically, any generator of S2 (M) can be represented by the following object: a smooth 2 2 principal G-bundle P → S and a smooth section φ : S → P ×G M. We denote the class of the G pair (P, φ) to be [P, φ] ∈ S2 (M).

2.1.2. Equivariant symplectic form and equivariant Chern numbers. The equivariant cohomology of M can also be computed using the equivariant de Rham complex Ω∗(M)G, dG. In Ω2(M)G, there is a distinguished closed form ω − µ, called the equivariant symplectic form, which represents an equivariant cohomology class. We are interested in the pairing h[ω − µ], [P, u]i ∈ R. It can be computed in the following way. Choose any smooth connection A on P . Then there exists an associated closed 2-form ωA on 2 P ×G M, called the minimal coupling form. If we trivialize P locally over a subset U ⊂ S , 1 such that A = d + α, α ∈ Ω (U, g) with respect to this trivialization, then ωA can be written as

∗ 2 ωA = π ω − d(µ · α) ∈ Ω (U × M). (2.6)

Then we have Z ∗ h[ω − µ], [P, u]i = u ωA (2.7) S2 which is independent of the choice of A. On the other hand, any G-invariant almost complex structure J on X makes TX an equivariant complex vector bundle. So we have the equivariant ﬁrst Chern class

G G 2 c1 := c1 (TM) ∈ HG(M; Z). (2.8)

This is independent of the choice of J. GAUGED FLOER HOMOLOGY 9

2.1.3. Kirwan maps. The cohomological Kirwan map is a map

∗ ∗ κ : H (M; R) → H (M; R). (2.9)

Here we take R-coeﬃcients for simplicity. It is easy to check that G κ([ω − µ]) = [ω] ∈ H2(M; R), κ(c1 ) = c1(T M) ∈ H2(M; R). (2.10) We deﬁne

G G G N2 (M) = ker[ω − µ] ∩ kerc1 ⊂ S2 (M),N2(M) = ker[ω] ∩ kerc1(T M) ⊂ S2(M), (2.11) and

G G Γ := S2 (M)/N2 (M). (2.12)

2.2. The spaces of loops and equivalence classes. Let Pe be the space of smooth contractible parametrized loops in M × g and a general element of Pe is denoted by 1 xe := (x, f): S → M × g. (2.13)

Let Pe be a covering space of Pe, consisting of triples x := (x, f, [w]) where xe = (x, f) ∈ P and [w] is an equivalence class of smooth extensions of x to the disk D. The equivalence relation is 1 described as follows. For each pair w1, w2 : D → M both bounding x : S → M, we have the continuous map

2 w12 := w1#(−w2): S → M (2.14) by gluing them along the boundary x. We deﬁne

G w1 ∼ w2 ⇐⇒ [w12] = 0 ∈ S2 (M). (2.15) Denote by LG := L∞G := C∞(S1,G) the smooth free loop group of G. Then for any point x0 ∈ M, we have the homomorphism

l(x0): π1(G) → π1(M, x0) (2.16) which is induced by mapping a loop t 7→ γ(t) ∈ G to a loop t 7→ γ(t)x0 ∈ M. For diﬀerent x1 ∈ M and a homotopy class of paths connecting x0 and x1, we have an isomorphism π1(M, x0) ' π1(M, x1); it is easy to see that l(x0) and l(x1) are compatible with this isomorphism. This means kerl(x0) ⊂ π1(G) is independent of x0. Then we deﬁne 1 LM G := γ : S → G | [γ] ∈ kerl(x0) ⊂ π1(G) . (2.17)

Let L0G ⊂ LM G be the subgroup of contractible loops in G. It is easy to see that LM G acts on Pe (on the right) by

Pe × LM G → Pe ∗ −1 −1 −1 (2.18) ((x, f), h) 7→ h (x, f)(t) = h(t) x(t), Adh(t)(f(t)) + h(t) ∂th(t) . Here the action on the second component can be viewed as the gauge transformation on the space of G-connections on the trivial bundle S1 × G. (For short, we denote by d log h the g-valued 1-form h−1dh, which is the pull-back by h of the left-invariant Maurer-Cartan form on G.) 10 GUANGBO XU

But LM G doesn’t act on Pe naturally; only the subgroup L0G does: for a contractible loop 1 h : S → G, extend h arbitrarily to h : D → G. The homotopy class of extensions is unique because −1 ∗ −1 π2(G) = 0 for any connected compact Lie group ([1]). Then the class of (h x, h f, [h w]) in Pe is independent of the extension. It is easy to see that the covering map Pe → Pe is equivariant with respect to the inclusion L0G → LM G. Hence it induces a covering

Pe /L0G → Pe/LM G. (2.19)

G 2.3. The deck transformation and the action functional. We now deﬁne an action of S2 (M) G 2 on Pe /L0G. Take a class A ∈ S2 (M) represented by a pair (P, u), where P → S is a principal 2 G-bundle and u : S → P ×G M is a section of the associated bundle. ∗ 2 2 Consider Un ' C ∪ {∞} ⊂ S as the complement of the south pole 0 ∈ S . Take an arbitrary trivialization φ : P |Un → Un × G, which induces a trivialization

φ : P ×G M|Un → Un × M. (2.20)

1 Then φ ◦ u is a map from Un to M and there exists a loop h : S → G and x ∈ M such that

lim φ ◦ u(reiθ) = h(θ)x. (2.21) r→0 Note that the homotopy class of h is independent of the trivialization φ and the choice of x. Now, for any element (x, f, [w]) ∈ Pe , ﬁnd a smooth path γ : [0, 1] → M such that γ(0) = w(0) 1 and γ(1) = x. Then deﬁne γh : S × [0, 1] → M by γ(θ, t) = h(θ)γ(t). 1 On the other hand, view D \{0}' (−∞, 0] × S . Consider the map

wh(r, θ) = h(θ)w(r, θ) and the “connected sum”:

u#e w := (φ ◦ u)#γh#wh : D → M (2.22) which extends the loop

xh(θ) = h(θ)x(θ). (2.23)

−1 Denote fh := Adhf − ∂th · h . Then we deﬁne the action by

h i G A#[x, f, [w]] = xh, fh, [u#e w], , ∀A ∈ S2 (M). (2.24)

On the other hand, there exists a morphism

G S2 (M) → kerl(x0) ⊂ π1(G) (2.25) which sends the homotopy class of [P, u] to the homotopy class of h : S1 → G where h is the one in (2.21). Then it is easy to see the following.

Lemma 2.5. The action (2.24) is well-deﬁned (i.e., independent of the representatives and choices) and is the deck transformation of the covering Pe /L0G → Pe/LM G. GAUGED FLOER HOMOLOGY 11

G Now by this lemma, we denote P := Pe /L0G /N2 , which is again a covering of P := Pe/LM G, with the group of deck transformations isomorphic to Γ. We will use x to denote an element in Pe or Pe /L0G and use [x] an element in P. We can deﬁne a 1-form BeH on Pe by Z T(x,f)Pe 3 (ξ, h) 7→ {ω (x ˙(t) + Xf − YHt , ξ(t)) + hµ(x(t)), h(t)i} dt ∈ R. (2.26) S1 Its pull-back to Pe is exact and one of the primitives is the following action functional on Pe : Z Z ∗ AeH (x, f, [w]) := − w ω + (µ(x(t)) · f(t) − Ht(x(t))) dt. (2.27) B S1

The zero set of the one-form BeH consists of pairs (x, f) such that

µ(x(t)) ≡ 0, x˙(t) + Xf(t)(x(t)) − YHt (x(t)) = 0. (2.28)

The critical point set of AeH are just the preimage of ZeroBeH under the covering Pe → Pe.

Lemma 2.6. AeH is L0G-invariant and BeH is LM G-invariant.

1 Proof. Take any h : S → G, extend it smoothly to some h : D → G. Then we see that −1 ∗ −1 −1 ∗ −1 (h w) ω = ω ∂x(h w), ∂y(h w) dxdy = w ω + d µ(h w) · d log h . (2.29) Also, we see that µ(h−1(t)x(t)) · Ad−1 f(t) + h(t)−1h0(t) = µ(x(t)) · f(t) + µ(h−1w) · d log h . (2.30) h(t) ∂D

By Stokes’ theorem and the G-invariance of Ht, we obtain the invariance of AeH . The LM G- invariance of BeH follows from equivariance of the involved terms in a similar way.

So we have the induced action functional AeH : Pe /L0G → R and it satisﬁes the following.

G Lemma 2.7. For any [x] = [x, f, [w]] ∈ Pe /L0G and any A ∈ S2 (M), we have

AeH (A#[x]) = AeH ([x]) − h[ω − µ],Ai. (2.31)

Proof. Use the same notation as we deﬁne the action A#[x], we see that

Z Z Z 0 Z ∗ ∗ whω = w ω + ds dtω (h∗∂sw, h∗X∂t log h(w)) D\{0} D −∞ S1 Z Z 0 Z 1 Z Z = w∗ω − d (µ(w) · d log h) = w∗ω − (µ(x(t)) − µ(w(0)))d log h. (2.32) D −∞ 0 D S1 Also Z Z ∗ γhω = − (µ(w(0)) − µ(x)) d log h. (2.33) S1×[0,1] S1 In the same way we can calculate Z Z (φ ◦ u)∗ ω = h[ω − µ], [P, u]i − µ(x)d log h. (2.34) S2\{0} S1 12 GUANGBO XU

So we have Z ∗ Z 0 −1 AeH (A#[x]) = − u#e w ω + µ(h(t)x(t)), Adh(t)f(t) − h (t)h(t) − Ht(h(t)x(t)) dt D S1 Z Z ∗ = −h[ω − µ],Ai − w ω + hµ(x(t)), f(t)i − Ht(x(t))dt = −h[ω − µ],Ai + AeH ([x]) (2.35) D S1

This lemma implies that AeH descends to a well-deﬁned function

AH : P → R. (2.36)

Our Floer theory will be formally a Morse theory of the pair (P, AH ). Before we move on to the chain complex, we see that AH is a Lagrange multiplier function associated to the action functional AH of the induced Hamiltonian H on the symplectic quotient M. Let Pe M be the space of contractible loops in M and let PM be pairs (x, [w]) where x ∈ Pe M 0 0 and w : D → M extends x;[w] = [w ] if (−w)#w is annihilated by both ω and c1(T M). Then for −1 any (x, [w]) ∈ PM , we can pull-back the principal G-bundle µ (0) → M to D. Any trivialization −1 (or equivalently a section s) of this bundle over D induces a map ws : D → µ (0) whose boundary 1 −1 restriction, denoted by x : S → µ (0), lifts x. Now, if (x, [w]) ∈ CritAH , i.e. 0 = x0(t) − Y (x(t)) = (π ) x0(t) − Y (x(t)) (2.37) Ht µ ∗ Ht 1 there exists a smooth function, fs : S → g such that 0 x (t) + Xfs(t)(x(t)) − YHt (x(t)) = 0. (2.38) Then this gives a map

ι : Pe → Pe /L0G M (2.39) (x, [w]) 7→ [xs, fs, [ws]] By the correspondence of the symplectic forms and Chern classes between upstairs and downstairs, we have

Proposition 2.8. The class [xs, fs, [ws]] is independent of the choice of the section s and only depends on the homotopy class of w. Moreover, it induces a map ι : PM , CritAH → (P, CritAH ) (2.40)

2.4. The Floer chain complex. For R = Z2, Z or Q, we consider the Novikov ring of formal power series over a base ring R to be the downward completion of R[Γ]: ( )

↓ X B ΛR := ΛR := λBq ∀K > 0, # {B ∈ Γ | h[ω − µ],Bi > K, λB 6= 0} < ∞ . (2.41) B∈Γ

We deﬁne the free ΛR-module generated by the set CritAH ⊂ P to be VCF\(M, µ; H;ΛR). We deﬁne an equivalence relation on VCF\(M, µ; H;ΛR) by [x] ∼ qB x0 ⇐⇒ B# x0 = [x] ∈ P. (2.42)

Denote by VCF (M, µ; H;ΛR) the quotient ΛR-module by the above equivalence relation, which is will be graded by a Conley-Zehnder type index which will be deﬁned later in Section4. GAUGED FLOER HOMOLOGY 13

2.5. Gradient ﬂow and symplectic vortex equation. Now we choose an S1-family of G- invariant, ω-compatible almost complex structures J on M, we assume that

f(x) ≥ c0 =⇒ Jt(x) = J(x) (2.43) where (f, J) is the convex structure which we assume to exist in Hypothesis 2.4. Then ω and Jt 2 defines a Riemannian metric on M, which induces an L -metric ge on the loop space LM. On the other hand, we fix a biinvariant metric on the Lie algebra g which induces a metric g2 on Lg; it also identifies g with g∗ and we use this identification everywhere in this paper without mentioning it. These choices induce a metric on P.

Then, it is easy to see that formally, the equation for the negative gradient ﬂow line of AH is the following equation for a pair (u, Ψ) : Θ → M × g

 ∂u ∂u  + J + X − Y = 0,  t Ψ Ht ∂s ∂t (2.44) ∂Ψ  + µ(u) = 0.  ∂s The equation is invariant under the action of LG. The action is deﬁned by

LG × Map (Θ,M × g) → Map (Θ,M × g) ∗ −1 −1 −1 (2.45) g (u, Ψ)(s, t) = g(t) u(s, t), Adg(t)Ψ(s, t) + g(t) ∂tg(t)

Definition 2.9. The energy for a flow line ue (or its Yang-Mills-Higgs functional) is defined to be 2 1 2 1 ∂Ψ 1 2 E (u) := E (u, Ψ) := kdu + (X − Y ) ⊗ dtk 2 + + kµ(u)k 2 . (2.46) e Ψ Ht L L 2 2 ∂s L2 2 The second equation of (2.44) is actually the symplectic vortex equation on the triple (P, u, Ψdt), where P → Θ is the trivial G-bundle, u is a section of P ×G M and Ψdt corresponds to the covariant derivative d + Ψdt. (For a detailed introduction to the symplectic vortex equation, see [3] or [25]). This is why we name our theory the vortex Floer homology. The connection d + Ψdt has already been put in the temporal gauge, i.e., it has no ds component. A more general equation on pairs (u, α), with α = Φds + Ψdt ∈ Ω1(Θ) ⊗ g is

 ∂u ∂u  + X + J + X − Y = 0,  Φ t Ψ Ht ∂s ∂t (2.47) ∂Ψ ∂Φ  − + [Φ, Ψ] + µ(u) = 0.  ∂s ∂t

We write the object (u, α) in the form of a triple ue = (u, Φ, Ψ). The above equation is invariant ∞ under the action by GΘ := C (Θ,G), which is deﬁned by

GΘ × Map (Θ,M × g × g) → Map (Θ,M × g × g)    −1  u g(s, t) u(s, t) (2.48) ∗    Ad−1 Φ(s, t) + g(s, t)−1∂ g(s, t)  g  Φ  (s, t) =  g(s,t) s  −1 −1 Ψ Adg(s,t)Ψ(s, t) + g(s, t) ∂tg(s, t) 14 GUANGBO XU

A solution ue = (u, Φ, Ψ) to (2.47) is called a generalized ﬂow line. The energy of a generalized ﬂow line is

1 1 2 1 2 E(u, Φ, Ψ) = kdu + X ⊗ ds + (X − Y ) ⊗ dtk 2 + kµ(u)k 2 + k∂ Ψ − ∂ Φ + [Φ, Ψ]k 2 . 2 Φ Ψ Ht L 2 L 2 s t L (2.49) It is easy to see that any smooth generalized ﬂow line is gauge equivalent via a gauge transformation in GΘ to a smooth ﬂow line in temporal gauge; and the energy is gauge independent.

2.6. Moduli space and the formal deﬁnition of the vortex Floer homology. We focus on solutions to (2.47) with ﬁnite energy. In Section3 we will show that, any such solution is gauge equivalent to a solution ue = (u, Φ, Ψ) such that there exists a pair xe± = (x±, f±) ∈ ZeroBeH with

lim Φ(s, t) = 0, lim (u(s, ·), Ψ(s, ·)) = x±. (2.50) s→±∞ s→±∞ e

Hence for any pair [x±] ∈ CritAH , we can consider solutions which “connect” them. We denote by

M ([x−], [x+]; J, H) (2.51) the moduli space of all such solutions, modulo gauge transformation.

In the appendix we will show that, if we assume that Ht vanishes for t in a nonempty open subset I ⊂ S1, and J a generic, “admissible” family of almost complex structures, then the space

M ([x−], [x+]; J, H) is a smooth manifold, whose dimension is equal to the difference of the Conley- Zehnder indices of [x±]. Moreover, assuming that the manifold M cannot have any nonconstant pseudoholomorphic spheres, we show that M ([x−], [x+]; J, H) is compact modulo breakings. Finally, there exist coherent orientations on different moduli spaces, which is similar to the case of ordinary Hamiltonian Floer theory (see [10]). Then, the signed counting of isolated gauged equivalence classes of trajectories in our case has exactly the same nature as in finite-dimensional Morse-Smale- Witten theory, which defines a boundary operator

δJ : VCF∗(M, µ; H;ΛZ) → VCF∗−1(M, µ; H;ΛZ). (2.52) The vortex Floer homology is then deﬁned

VHF∗ (M, µ; J, H;ΛZ) = H (VCF∗(M, µ; H;ΛZ), δJ ) . (2.53) Moreover, for a diﬀerent choice of the pair (J 0,H0), we can use continuation principle to prove 0 that the chain complex (VCF∗(M, µ; H ;ΛZ), δJ0 ) is chain homotopic to (VCF∗(M, µ; H;ΛZ); δJ ). There is a canonical isomorphism between the homologies, and we denote the common homology group by VHF∗(M, µ;ΛZ). The details are given in Section6 and the appendix.

3. Asymptotic behavior of the connecting orbits

In this section we analyze the asymptotic behavior of solutions ue = (u, Φ, Ψ) to (2.47) which has ﬁnite energy and for which u(Θ) has compact closure in M. We call such a solution a bounded b solution. We denote the space of bounded solutions by MfΘ. We can also consider the equation on the half cylinder Θ or Θ and denote the spaces of bounded solutions over Θ by Mb . + − ± fΘ± The main theorem of this section is GAUGED FLOER HOMOLOGY 15

Theorem 3.1. (1) Any (u, Φ, Ψ) ∈ Mb is gauge equivalent (via a smooth gauge transforma- fΘ± tion g :Θ → G) to a solution (u0, Φ0, Ψ0) ∈ Mb such that there exist x = (x , f ) ∈ ± fΘ± e± ± ± ZeroBeH and 0 0 lim u (s, ·), Ψ (s, ·) = x±, lim Φ(s, ·) = 0 (3.1) s→±∞ e s→±∞ uniformly for t ∈ S1. b (2) There exists a compact subset KH ⊂ M such that for any (u, Φ, Ψ) ∈ MfΘ, we have u(Θ) ⊂ KH .

We will prove (1) for u ∈ Mb in temporal gauge, i.e., Φ ≡ 0 and the case for Θ is the same. e fΘ+ − Then (2) follows from a maximum principle argument, given at the end of this section. The proof is based on estimates on the energy density, which has been given by others in several diﬀerent settings (see [2], [16]). The only possibly new ingredient is that we have a nonzero Hamiltonian here.

1 3.1. Covariant derivatives. The S -family of metrics gt := ω(·,Jt·) induces a metric connection ∇ on the bundle u∗TM. Moreover, we deﬁne

∇A,sξ = ∇sξ + ∇ξXΦ, ∇A,tξ = ∇tξ + ∇ξXΨ. (3.2) Also, on the trivial bundle Θ × g, deﬁne the covariant derivative

∇A,sθ = ∇sθ + [Φ, θ], ∇A,tθ = ∇tθ + [Ψ, θ]. (3.3) ∗ We denote by ∇A the direct sum connection on u TM × g. Note that it is compatible with respect to the natural metric on this bundle.

Deﬁne the g-valued 2-form ρt on M by

hρ(ξ1, ξ2), ηit = h∇ξ1 Xη, ξ2it = −h∇ξ2 Xη, ξ1it, ξi ∈ T M, η ∈ g. (3.4) We list several useful identities of this covariant derivative. The reader may refer to [16] for details.

Lemma 3.2. For ue in temporal gauge, we have the following equalities

∇A,sXη − X∇A,sη = ∇∂suXη, ∇A,tXη − X∇A,tη = ∇∂tu+XΨ Xη. (3.5)

∇A,s (∂tu + XΨ) − ∇A,t∂su = X∂tΨ. (3.6)

∇A,s (dµ · Jξ) − dµ · J (∇A,sξ) = ρ(∂su, ξ), ∇A,t (dµ · Jξ) − dµ · J (∇A,tξ) = ρ(∂tu + XΨ, ξ). (3.7) On the other hand, since our equation is perturbed by a Hamiltonian H, we consider another t covariant derivative which takes H into account. Let φH be the Hamiltonian isotopy deﬁned by Ht and H t −1 t Jt := dφH ◦ Jt ◦ dφH , t ∈ R be the 1-parameter family of almost complex structures. They are still compatible with ω and H hence deﬁnes a family of metric gt , with induced inner product denoted by h·, ·iH,t. We denote the induced metric connection on u∗TM by ∇H . 16 GUANGBO XU

We think (u, Ψ) as a map from R × R → M × g, periodic in the second variable. Then we have ∗ a well-deﬁned connection D on u TM → R × R, given by H H H H (Dsξ)(s, t) = ∇s ξ (s, t), (Dtξ)(s, t) = ∇t ξ + ∇ξ XΨ − ∇ξ YHt . (3.8)

∗ Lemma 3.3. D is compatible with the inner product h·, ·iH,t on u TM.

Proof. By direct calculation, we see for ξ, η ∈ Γ(u∗TM),

∂shξ, ηiH,t = hDsξ, ηiH,t + hξ, DsηiH,t,

D E dgt gt gt gt gt ∂thξ, ηiH,t = (ξ, η) + ∇t ξ + ∇ξ XF , η + ξ, ∇t η + ∇η XF dt H,t Ht D E gt gt gt gt = − LYH gt (ξ, η) + ∇t ξ + ∇ξ XF , η + ξ, ∇t η + ∇η XF t H,t Ht

= hDtξ, ηiH,t + hξ, DtηiH,t .

3.2. Estimate of the energy density.

Lemma 3.4. There exist positive constants c1 and c2 depending only on (X, ω, J, µ, Ht) and the subset Ku, such that for any ﬂow line (u, 0, Ψ) in temporal gauge, we have

2 4 ∆ |∂su|H,t ≥ −c1|∂su|H,t − c2. (3.9)

Proof. First we see that

1 2 2 2 2 ∆ |∂su|H,t = ∂s + ∂t |∂su|H,t = ∂s hDs∂su, ∂suiH,t + ∂thDt∂su, ∂suiH,t 2 (3.10) 2 2 2 2 = |Ds∂su|H,t + |Dt∂su|H,t + Ds + Dt ∂su, ∂su H,t . 2 ∗ 2 ∗ We denote vs = ∂su ∈ Γ(R , u TM) and vt = ∂tu + XΨ − YHt ∈ Γ(R , u TM). Hence vs = −Jtvt. Then we have the following computation

Dsvt − Dtvs (3.11) H H H =∇s (∂tu + XΨ − YHt ) − ∇t ∂su + ∇∂su (−XΨ + YHt ) = X∂sΨ = −Xµ(u).

Dsvs + Dtvt

=Ds(−Jtvt) + Dt(Jtvs) H H H = − ∇s (Jtvt) + ∇t (Jtvs) + ∇Jvs (XΨ − YHt ) H H H = − ∇ (Jtvt) + ∇ (Jtvs) + [Jtvs,XΨ − YH ] + ∇ (Jtvs) s t t XΨ−YHt H H ˙ = − ∇s (Jvt) + ∇vt (Jtvs) + Jtvs + [Jtvs,XΨ − YHt ] H H H H ˙ = − (∇vs Jt)vt − Jt∇s vt + (∇vt Jt)vs + Jt∇vt vs + Jtvs + Jt [vs,XΨ] + [YHt ,Jtvs] = − (∇H J )v + (∇H J )v − J [v , v ] − JX + J˙ v + J [v ,X ] + J [Y , v ] + (L J )v vs t t vt t s t s t ∂sΨ t s t s Ψ t Ht s YHt t s = − (∇H J )v + (∇H J )v + (L J )v + J X + J˙ v . vs t t vt t s YHt t s t µ t s (3.12) GAUGED FLOER HOMOLOGY 17

On the other hand, for any (s, t) ∈ Σ, any ξ ∈ Tu(s,t)M, we extend ξ and vs(s, t) to be G-invariant H vector ﬁelds locally. Then for the Riemann curvature tensor associated to Jt , we have

H H H H H H H H H R (vs,XΨ)ξ = ∇v ∇X ξ − ∇X ∇v ξ − ∇ ξ = ∇v ∇ξ XΨ − ∇ H XΨ. (3.13) s Ψ Ψ s [vs,XΨ] s ∇vs ξ Hence

(DsDt − DtDs) ξ

H H H H H H = ∇ ∇ ξ + ∇ (XΨ − YH ) − ∇ (∇ ξ) − ∇ H (XΨ − YH ) s t ξ t t s ∇s ξ t H H H H d H = R (vs, ∂tu)ξ + ∇s ∇ξ (XΨ − YHt ) − ∇ H (XΨ − YHt ) − ∇s ξ ∇s ξ dt H H H H H d H = R (vs, ∂tu + XΨ)ξ − ∇s ∇ξ YHt + ∇ H YHt + ∇ξ X∂sΨ − ∇s ξ ∇s ξ dt H H H H H d H = R (vs, vt + YHt )ξ − ∇s ∇ξ YHt + ∇ H YHt − ∇ξ Xµ − ∇s ξ. (3.14) ∇s ξ dt

The third equality above uses (3.13). Then we denote

2 2 Ds vs + Dt vs = Ds (Dsvs + Dtvt) + (DtDs − DsDt) vt − Dt (Dsvt − Dtvs) =: Q1 + Q2 + Q3.

By (3.12),

D H H H ˙ E hQ1, vsiH,t = ∇s − ∇v Jt vt + ∇v Jt vs + LYH Jt vs + JtXµ + Jtvs , vs s t t H,t 3 2 4 2 H H ≥ −C1 |vs|H,t + |vs|H,t + |vs|H,t + |vs|H,t ∇s vs H,t + |vs|H,t ∇s vs H,t (3.15) for some C1 > 0. Here we used the fact that µ and dµ are uniformly bounded because u(Θ+) has compact closure.

By (3.14), for some C2 > 0, we have

H H H H H d H hQ2, vsiH = −R (vs, vt + YH,t)vt + ∇ ∇ YH − ∇ H YH + ∇ Xµ + ∇ vt, vs t s vt t ∇s vt t vt s dt H,t 4 3 2 H ≥ −C2 |vs|H,t + |vs|H,t + |vs|H,t + |vs|H,t ∇s vs H,t . (3.16)

By (3.11), for some C3 > 0, we have

hQ3, vsiH,t

= hDtXµ, vsiH,t D H H E H = ∇t Xµ + ∇X (XΨ − YHt ), vs = Xdµ·∂tu + ∇v Xµ + [Xµ,XΨ − YHt ], vs (3.17) µ H,t t = X + ∇H X − X , v = X − X + ∇H X − X , v dµ·∂tu vt µ [µ,Ψ] s dµ·vt dµ·XΨ vt µ [µ,Ψ] s H,t H 2 =hXdµ·vt + ∇vt Xµ, vsiH,t ≥ −C3|vs|H,t. 18 GUANGBO XU

Hence for some C4 > 0 and c1, c2 > 0, we have

1 ∆|v |2 = |D v |2 + |D v |2 + hQ + Q + Q , v i ≥ |∇gt v |2 + hQ + Q + Q , v i 2 s H s s H t t H 1 2 3 s H s s H 1 2 3 s H 4 2 ! gt 2 X i X i gt 4 ≥ |∇s vs|H − C4 |vs|H + |vs|H |∇s vs|H ≥ −c1|vs|H − c2. (3.18) i=2 i=1

Now we consider the other part of the energy density. We have the following calculation: 1 ∆ |µ(u)|2 = |∇ µ|2 + |∇ µ|2 + h∇ ∇ µ + ∇ ∇ µ, µi. (3.19) 2 A,s A,t A,s A,s A,t A,t And (see [16, Lemma C.2])

∇A,s∇A,sµ(u) + ∇A,t∇A,tµ(u)

=∇A,sdµ · vs + ∇A,tdµ · vt = ∇A,t (dµ · Jvs) − ∇A,s (dµ · Jvt) (3.20) = − 2ρ(vs, vt) + dµ · Jt (∇A,tvs − ∇A,svt) + dµ J˙tvs =dµ · JtXµ + J˙tvs − 2ρ(vs, vt).

Since u(Θ) has compact closure, we may assume that supu(Θ) |ρ| ≤ cu. Hence there exists c3, c4 > 0 such that

2 4 ∆ |µ(u)| ≥ −c3 − c4 |vs| . (3.21)

3.3. Decay of energy density. To proceed, we quote the following lemma (cf. [32, Page 12]).

2 Lemma 3.5. Let Ω ⊂ R be an open subset containing the origin and e is deﬁned over Ω. Suppose it satisﬁes ∆e ≥ −A − Be2, then Z π 8 Z Ar2 e ≤ =⇒ e(0) ≤ 2 e + . (3.22) Br(0) 16B πr Br(0) 4

2 2 Apply the above lemma to the function |vs|H,t +|µ(u)| and note that diﬀerent norms are actually equivalent, we see

Proposition 3.6. If (u, Ψ) satisﬁes the equation (2.44) such that u(Θ) has compact closure and E(u, Ψ) < ∞, then the energy density

2 2 ∂u ∂Ψ + ∂s ∂s converges to 0 as s → ±∞, uniformly in t. GAUGED FLOER HOMOLOGY 19

3.4. Approaching to periodic orbits.

Proposition 3.7. Any (u, 0, Ψ) ∈ Mb is gauge equivalent to a solution to (2.47) (v, Φ, Ψ) on fΘ+ Θ+ with the following properties 0 (1)( v, Ψ)|{s}×S1 converges in C to an element of ZeroBeH as s → +∞; (2) lim Φ(s, ·) = 0 uniformly in t. s→+∞ Proof. By the above proposition, we know that ﬁnite energy implies that µ(u(s, ·)) → 0 as s → +∞. ∗ ∗ Let g ⊂ g be the -open ball of the origin. Then there is an equivariant symplectic diﬀeomorphism

−1 ∗ −1 ∗ U := µ (g ) → µ (0) × g . (3.23)

Then µ|U is just the projection of the right hand side of (3.23) onto the second factor (see for example [18]). Let πµ be the projection on to the first factor, then we define the almost complex structure ∗ J0,t := πµ Jt µ−1(0) and the vector field ∗ Y0,Ht := πµ YHt µ−1(0) .

Then there exists K1 > 0 depending on (M, ω, J, µ, Ht) such that

kJ0,t(x) − Jt(x)k ≤ K1|µ(x)|, kY0,Ht (x) − YHt (x)k ≤ K1|µ(x)|, ∀x ∈ U.

−1 ∗ −1 We denote πµ : µ (g ) → M the composition of πµ with the projection µ (0) → M. For (u, 0, Ψ) ∈ Mb , we have proved that µ(u) converges to 0 uniformly as s → +∞. Hence for fΘ+ N 1 N N := N() suﬃciently large, u(s, ·) maps Θ+ := [N, +∞) × S into U. So on Θ+ , we have

∂su + J0,t (∂tu + XΨ(u) − Y0,Ht (u)) = (J0,t − Jt)(∂tu + XΨ(u) − Y0,Ht ) + Jt (YHt − Y0,Ht ) . (3.24)

N N Denoting u := πµ ◦ u :Θ+ → M and applying (πµ)∗ to the above equality, we see on Θ+ , ∂ u + J ∂ u − Y (u) ≤ K (3.25) s t t Ht 2 for some constant K2. Here J t is the induced almost complex structure on M. Hence for s ≥ N, the family of loops u(s, ·) in the quotient will be close (in C0) to some 1-periodic orbits γ : S1 → M of Y . Ht −1 We take a lift p ∈ µ (0) with ππµ(p) = γ(0). We see that there exists a unique gp ∈ G such that

1 φH p = gpp. (3.26)

Suppose gp = exp ξp, ξp ∈ g. It is easy to see that the loop

t (x(t), f(t)) := exp(−tξp)φH (p), ξp (3.27) is an element of ZeroBeH . We will construct a gauge transformation ge on Θ+ and show that ∗ ge (u, 0, Ψ) satisﬁes the condition stated in this proposition. 20 GUANGBO XU

2n−2k −1 Take a local slice of the G-action near p. That is, an embedding i : Bδ → µ (0) where 2n−2k 2n−2k Bδ is the δ-ball in R such that i(0) = p and (y, g) 7→ g(i(y)) is a diﬀeomorphism from 2n−2k Bδ × G onto its image. Denote u(s, t) = (v(s, t), ξ(s, t)) ∈ µ−1(0) × g∗ with respect to the decomposition (3.23). Then for s large enough, there exists a unique g(s) ∈ G such that

2n−2k g(s)v(s, 0) ∈ i Bδ , g(s)v(s, 0) → p. (3.28)

Moreover, by the fact that |∂su| converges to zero, we see

−1 lim g(s) g˙(s) = 0. (3.29) s→+∞

Deﬁne hs(t) ∈ G by ∂h (t) h (0) = 1, h (t)−1 s = Ψ(s, t). s s ∂t Then by the fact that lims→+∞ |∂sΨ| = 0 we see that

lim |∂s log hs(t)| = 0. (3.30) s→∞ Thus we have −1 d gpp, g(s)hs(1)g(s) p 1 1 ≤ d gpp, φH g(s)v(s, 0) + d φH g(s)v(s, 0), g(s)hs(1)v(s, 0) −1 + d g(s)hs(1)v(s, 0), g(s)hs(1)g(s) p (3.31) 1 1 = d gpp, φH g(s)v(s, 0) + d φH v(s, 0), hs(1)v(s, 0) + d (g(s)v(s, 0), p)

=: d1(s) + d2(s) + d3(s). Here d is the G-invariant distance function induced by an invariant Riemannian metric. By (3.26) and (3.28), we have d1(s) + d3(s) → 0. By the decay of energy density, i.e.,

lim sup |∂tu + XΨ − YHt | = 0, s→+∞ t we have d2(s) → 0. Hence we have −1 lim d gpp, g(s)hs(1)g(s) p = 0. (3.32) s→+∞ Since the G-action on µ−1(0) is free, we have

−1 lim g(s)hs(1)g(s) = gp (3.33) s→+∞ Then by (3.33), there exists a continuous curve ξ(s) ∈ g deﬁned for large s, such that

−1 g(s)hs(1)g(s) = exp ξ(s), lim ξ(s) = ξp. (3.34) s→+∞ Then apply the gauge transformation

−1 −1 ge(s, t) = hs(t) g(s) exp(tξ(s)) (3.35) to the pair (u, 0, Ψ), we see

∗ −1 ∗ (ge u)(s, 0) = ge(s, 0) (u(s, 0)) = g(s)u(s, 0) → p, ge (Ψdt) = ξ(s)dt + η(s, t)ds (3.36) GAUGED FLOER HOMOLOGY 21 where ∂g η = ∂ log g = g−1 e. s e e ∂s The fact that lims→+∞ kηk = 0 follows from (3.29) and (3.30). Hence ∗ t lim (g u)| 1 = exp(−tξp)φ p, ξp ∈ ZeroBeH . s→+∞ e {s}×S H

Deﬁnition 3.8. Let xe± := (x±, f±) ∈ ZeroBeH . We denote b Mf(x−, x+) := Mf(x−, x+; J, H) := (u, Φ, Ψ) ∈ Mf | lim (u, Φ, Ψ)|{s}×S1 = (x±, 0) . (3.37) e e e e Θ s→±∞ e

For x± = (xe±, [w±]) ∈ CritAeH which projects to xe± via CritAeH → ZeroBeH , we deﬁne n o Mf(x−, x+) := Mf(x−, x+; J, H) := (u, Φ, Ψ) ∈ Mf(xe−, xe+) | [u#w−] = [w+] . (3.38) Then it is easy to deduce the following energy identity for which we omit the proof.

Proposition 3.9. Let x± ∈ CritAeH . Then for any (u, Φ, Ψ) ∈ Mf(x−, x+), we have

E (u, Φ, Ψ) = AeH (x−) − AeH (x+) . (3.39)

3.5. Convexity and uniform bound on ﬂow lines. We will show in this subsection the following

b Proposition 3.10. There exists a compact subset KH ⊂ M such that for any (u, Φ, Ψ) ∈ MfΘ, u(Θ) ⊂ KH .

Proof. The proof is to use maximum principle as in [2, Subsection 2.5]. We claim this proposition is true for −1 KH = SuppH ∪ f ([0, c1]) where ( ) c1 = max c0, sup f(x) (3.40) |µ(x)|≤1 where c0 is the one in Hypothesis 2.4. b Suppose the statement is not true. Then there exists a solution ue = (u, Φ, Ψ) ∈ MfΘ which violates this condition and (s0, t0) ∈ Θ such that u(s0, t0) ∈/ SuppH and f(u(s0, t0)) > c1. On the other hand, by the previous results, we know that lim µ(u(s, t)) = 0 so lim f(u(s, t)) ≤ c1. s→±∞ s→±∞ Hence f(u) achieves its maximum at some point of Θ. As in the proof of [2, Lemma 2.7], we see that −1 f(u) is subharmonic on u (M \ KH ) and hence f(u) must be constant. However, this contradicts with the fact that lim f(u(s, t)) ≤ c1. s→±∞

4. Fredholm theory In this section we investigate the inﬁnitesimal deformation theory of solutions to our equation (modulo gauge). For a similar treatment of a relevant situation, the reader may refer to [4]. 22 GUANGBO XU

4.1. Banach manifolds, bundles, and local slices. First we ﬁx two loops xe± ∈ ZeroBeH . For k,p any k ≥ 1, p > 2, we consider the space of Wloc -maps ue := (u, Φ, Ψ) : Θ → M × g × g, such that k,p k,p Φ ∈ W (Θ, g) and (u, Ψ) is asymptotic to xe± = (x±, f±) at ±∞ in W -sense. Then this is a Banach manifold, denoted by

k,p k,p Be := Be (xe−, xe+). (4.1) k,p The tangent space at any element ue ∈ Be is the Sobolev space T Bk,p = W k,p (Θ, u∗TM ⊕ g ⊕ g) . (4.2) ue e t We denote by expg the exponential map of M ×g×g, where the Riemannian metric on M is ω(·,Jt·) which is t-dependent. Then the map ξ 7→ expt ξ is a local diﬀeomorphism from a neighborhood of e gue e 0 ∈ T Bk,p and a neighborhood of u in Bk,p. ue e e e Then consider a pair x± = (x±, f±, [w±]) ∈ CritAeH with xe± = (x±, f±) ∈ ZeroBeH . We deﬁne k,p n k,p o Be (x−, x+) := ue = (u, Φ, Ψ) ∈ Be (xe−, xe+) | [w−#u] = [w+] . (4.3)

k+1,p k+1,p Let G0 be the space of Wloc -maps g :Θ → G which is asymptotic to the identity of G at k+1,p ±∞. Then this is a Banach Lie group. The gauge transformation extends to a free G0 -action k,p k,p on Be (xe−, xe+) (resp. Be (x−, x+)), because the symplectic quotient M is a free quotient. Then this makes the quotient

k,p k,p k+1,p k,p k,p k+1,p B (xe−, xe+) := Be (xe−, xe+)/G resp. B (x−, x+) := Be (x−, x+)/G0 (4.4) a Banach manifold. Indeed, to see this we have to construct local slices of the Gk+1,p-action. For k,p k,p any ue ∈ Be (whose image in B is denoted by [ue]), consider the operator ∗ k,p k−1,p d : TuBe → W (Θ, g) 0 e (4.5) (ξ, φ, ψ) 7→ −dµ(Jtξ) − ∂sφ − [Φ, φ] − ∂tψ − [Ψ, ψ], which is the formal adjoint of the inﬁnitesimal Gk+1,p-action. Then as in gauge theory, we have a natural identiﬁcation

T Bk,p ' kerd∗ (4.6) [ue] 0 (where the orthogonal complement is taken with respect to the L2-inner product) and the expo- t nential map expg induces a local diﬀeomorphism h i kerd∗ 3 ξ 7→ expt ξ ∈ Bk,p. (4.7) 0 e gue e

0 ∗ If we have g± ∈ L0G, and x± = g±x± ∈ CritAeH , then the pair (g−, g+) extends to a smooth k,p k,p 0 0 gauge transformation on Θ which identiﬁes Be (x−, x+) with Be (x−, x+). Then, with abuse of k,p notation, if x± ∈ CritAeH /L0G, then we can denote by B (x−, x+) to be the common quotient space. Finally, for two pairs [x±] ∈ CritAH , we deﬁne

k,p [ k,p B ([x−], [x+]) := B (y−, y+), (4.8)

y±∈CritAeH /L0G, [y±]=[x±] which is a discrete union of Banach manifolds. GAUGED FLOER HOMOLOGY 23

k,p k−1,p Over Be (x−, x+), we have the smooth Banach space bundle Ee (x−, x+), whose ﬁbre over ue is the Sobolev space

k−1,p Ee := W k−1,p (u∗TM ⊕ g) . (4.9) ue k+1,p k−1,p The G0 -action makes Ee an equivariant bundle, hence descends to a Banach space bundle

k−1,p k,p k−1,p k,p E (x−, x+) → B (x−, x+) or E ([x−], [x+]) → B ([x−], [x+]) . (4.10)

Moreover, the H-perturbed vortex equation (2.47) gives a section

k,p k−1,p Fe : Be (x−, x+) → Ee (x−, x+) (4.11) which is Gk+1,p-equivariant. So it descends to a section

k,p k−1,p F : B ([x−], [x+]) → E ([x−], [x+]) .

−1 Then we see that Mf(x−, x+; J, H) is the intersection of Fe (0) with smooth objects. We deﬁne

−1 M ([x−], [x+]; J, H) = F (0), (4.12) whose elements, by the standard regularity theory about symplectic vortex equation (see [2, The- orem 3.1]), all have smooth representatives. Therefore M ([x−], [x+]; J, H) is independent of k, p. The linearization of Fe at ue is ! ∇ ξ + (∇ J )(∂ u + X − Y ) + J (∇ ξ − ∇ Y ) + X + JX D := dF :(ξ, φ, ψ) 7→ A,s ξ t t Ψ Ht t A,t ξ Ht φ ψ eue eue ∂sψ + [Φ, ψ] − ∂tφ − [Ψ, φ] + dµ(ξ) (4.13)

Hence the linearization of F, under the isomorphism (4.6), is the restriction of D to (kerd∗)⊥. eue 0 We deﬁne the augmented linearized operator

∗ k,p k−1,p k−1,p Du := Deu ⊕ d0 : TuBe → Ee ⊕ W (Θ, g) . (4.14) e e e ue It is a standard result that the Fredholm property of dF is equivalent to that of D for any [ue] ue representative ue. Hence in the remaining of the section we will study the Fredholm property of the augmented operator.

4.2. Asymptotic behavior of D . Up to an L G-action we can choose representatives x = ue 0 ± k,p (x±, f±, [w±]) such that f± are constants θ± ∈ g. Take any ue = (u, Φ, Ψ) ∈ Be (x−, x+). For ξ := (ξ, ψ, φ) ∈ T Bk,p(x , x ), deﬁne J(ξ, ψ, φ) = (Jξ, −φ, ψ). Here ψ is the variation of Ψ e ue e − + e and φ is the variation of Φ. Then             ξ ξ ∇A,tξ − ∇ξYH,t 0 JLu Lu ξ ξ D  ψ  = ∇  ψ +J  ∇ ψ + dµ 0 0   ψ +q(s, t)  ψ  ue   A,s   e A,t        ∗ φ φ ∇A,tφ Lu 0 0 φ φ        ∇s ∇t ξ        =:  ∂s  + Je ∂t  + R(s, t) + q(s, t)  ψ  . (4.15) ∂s ∂t φ 24 GUANGBO XU

Here q(s, t) is a linear operator such that lim q(s, t) = 0; and s→±∞   Jt∇(Xθ± − YHt ) JtLu Lu Rx (t) := lim R(s, t) =  dµ 0 −adθ±  . (4.16) e± s→±∞   ∗ Lu adθ± 0 ∗ ∗ Here Lu : g → u TM is the inﬁnitesimal action along the image of u and Lu is its dual. The following result is implied by Hypothesis 2.3.

Proposition 4.1. xe := (x, θ) ∈ ZeroBeH , then for all t-dependent, G-invariant, ω-compatible almost complex structure Jt, the self-adjoint operator L2 S1, x∗TM ⊕ g ⊕ g → L2 S1, x∗TM ⊕ g ⊕ g         ξ ∇ ξ t (4.17)  ψ  7→ J  ∂  + R (t)  ψ  .    e t  xe    φ ∂t φ has zero kernel. Proof. Suppose (ξ, ψ, φ)T is in the kernel, which means

Jt∇tξ + Jt∇ξ (Xθ − YHt ) + Xφ + JtXψ = 0, (4.18) dφ − + dµ(ξ) − [θ, φ] = 0, (4.19) dt dψ + dµ(J ξ) + [θ, ψ] = 0. (4.20) dt t Apply dµ ◦ Jt to (4.18), we get

dµ(JtXφ) = dµ(∇tξ + ∇ξ(Xθ − YHt )). Hence for any η ∈ g, d d hdµ(ξ), ηi = ω(X , ξ) dt g dt η

=ω([YHt − Xθ,Xη], ξ) + ω(Xη, ∇tξ − ∇ξ(YHt − Xθ)) (4.21) =ω(X[θ,η], ξ) + hdµ(JXφ), ηig

=hdµ(ξ), [θ, η]ig + hdµ(JXφ), ηig

=hdµ(JXφ) − [θ, dµ(ξ)], ηig. Therefore, d dµ(ξ) = dµ(JX ) − [θ, dµ(ξ)]. (4.22) dt φ Then by (4.19),

dµ(JtXφ) d = dµ(ξ) + [θ, dµ(ξ)] dt (4.23) d dφ dφ = + [θ, φ] + θ, + [θ, φ] dt dt dt =φ00 + 2 θ, φ0 + [θ, [θ, φ]]. GAUGED FLOER HOMOLOGY 25

1 Suppose kφk takes its maximum at t = t0 ∈ S . Then for t ∈ (t0 − , t0 + ), deﬁne φe(t) = 00 Ade(t−t0)θ φ(t). Then the right hand side of (4.23) is equal to Ade(t0−t)θ φe (t). Hence at t = t0, 1 d2 1 d2 2 D E 2 0 ≥ kφk2 = φ = φ00, φ + φ0 2 2 e e e e 2 dt 2 dt g (4.24) 2 2 = dµ(J X ), φ(t ) + φ0(t ) = X 2 + φ0(t ) . t φ(t0) 0 e 0 φ(t0) e 0 −1 Hence Xφ ≡ 0, which implies φ ≡ 0 and by (4.19), ξ is tangent to µ (0). ∗ −1 ⊥ ∗ −1 Now x T µ (0) = Et ⊕ Lxg, where Et = (Lxg) ∩ x T µ (0) and the orthogonal complement is taken with respect to the Riemannian metric g = ω(·,Jt·). Then with respect to this (G-invariant) decomposition, write ⊥ ξ = ξ (t) + Xη(t).

Then take the Et-component of (4.18), and use the nondegeneracy assumption on the induced ⊥ Hamiltonian Ht on the symplectic quotient, we see that ξ ≡ 0. The only equations left is   ∇tXη(t) + ∇Xη(t) (Xθ − YHt ) + Xψ = 0, 0  ψ + [θ, ψ] + dµ(JXη) = 0. The ﬁrst equation is equivalent to η0 + [θ, η(t)] + ψ = 0. Hence 00 0 η (t) + 2 θ, η (t) + [θ, [θ, η]] = dµ(JtXη). This can be treated similarly as (4.23), using maximum principle, which shows that η ≡ 0 and hence ψ ≡ 0. k,p Corollary 4.2. Under Hypothesis 2.3, for any xe± ∈ ZeroBeH and any ue ∈ Be (xe−, xe+), the augmented linearized operator D is Fredholm for any k ≥ 1, p ≥ 2. ue Corollary 4.3. There exists δ = δ(xe±) > 0 such that, for any ue = (u, Φ, Ψ) ∈ Mf(xe−, xe+; J, H) and any ξ ∈ kerD , there exists c > 0 such that e ue

−δ|s| ξe(s, t) ≤ ce . (4.25) b In particular, if (u, 0, Ψ) ∈ MfΘ, then |∂su| and |∂sΨ| decay exponentially.

Proof. The ﬁrst part is standard, see for example [32, Lemma 2.11]. For a solution ue = (u, 0, Ψ) in temporal gauge, by the translation invariance of the equation (2.44), we see that ξe = (∂su, βs = 0, β = ∂ F ) ∈ kerdF . Moreover, t s eue ∗ ∗ ∗ −d0ξe = − d0(∂su, 0, ∂sΨ) = ∂t∂sΨ + Lu(∂su) + [Ψ, ∂sΨ]

= − ∂t(µ(u)) + dµ(Jt∂su) + [Ψ, ∂sΨ] (4.26) = − dµ(∂tu) + dµ(Jt∂su) + [Ψ, ∂sΨ]

= dµ (XΨ − YHt ) − [Ψ, µ] = 0. This implies that ξ ∈ kerD . Choose a smooth gauge transformation g :Θ → G such that e ue ∗ ∗ g u satisﬁes the asymptotic condition of Proposition 3.7. Then g ξ ∈ kerD ∗ , which decays e e g ue exponentially. So does ξe. 26 GUANGBO XU

4.3. The Conley-Zehnder indices. In this subsection we deﬁne a grading on the set CritAH , which is analogous to the Conley-Zehnder index in usual Hamiltonian Floer theory, and we will call it the same name. For the induced Hamiltonian system on the symplectic quotient M, we have the usual Conley- Zehnder index

CZ : CritAH → Z. (4.27) We prove the following theorem

Theorem 4.4. There exists a function

CZ : CritAH → Z (4.28) satisfying the following properties

(1) For the embedding ι : CritAH → CritAH , we have CZ ◦ ι = CZ; (4.29)

(2) For any B ∈ Γ and [x] ∈ CritAH we have G CZ (B#[x]) = CZ ([x]) − 2c1 (B). (4.30) k,p (3) For [x±] = [x±, f±, [w±]] ∈ CritAH and [ue] ∈ B ([x−], [x+]) with [ue#w−] = [w+], we have ind dF = CZ ([x ]) − CZ ([x ]) . (4.31) [ue] − + We ﬁrst review the notion of Conley-Zehnder index in Hamiltonian Floer homology. Let A : [0, 1] → Sp(2n) be a continuous path of symplectic matrices such that

A(0) = I2n, det (A(1) − I2n) 6= 0. (4.32) We can associate an integer CZ(A) to A, called the Conley-Zehnder index. We list some properties of the Conley-Zehnder index below which will be used here (see for example [31]). (1) For any path B : [0, 1] → Sp(2n), we have CZ(BAB−1) = CZ(A); (2) CZ is homotopy invariant; (3) If for t > 0, A(t) has no eigenvalue on the unit circle, then CZ(A) = 0;

(4) If Ai : [0, 1] → Sp(2ni) for n = 1, 2, then CZ(A1 ⊕ A2) = CZ(A1) + CZ(A2); (5) If Φ : [0, 1] → Sp(2n) is a loop with Φ(0) = Φ(1) = Id, then

CZ(ΦA) = CZ(A) + 2µM (Φ) (4.33)

where µM (Φ) is the Maslov index of the loop Φ. With this algebraic notion, in the usual Hamiltonian Floer theory one can deﬁne the Conley- Zehnder indices for nondegenerate Hamiltonian periodic orbits. In our case, the induced Hamil- tonian Ht : M → R has the usual Conley-Zehnder index

CZ : CritAH → Z. (4.34)

Then, for each x = (x, f, [w]) ∈ CritAeH , the homotopy class of extensions [w] induces a homotopy class of trivializations of x∗TM over S1. With respect to this class of trivialization, the operator

(4.17) is equivalent to an operator J0∂t + A(t), which deﬁnes a symplectic path. We deﬁne the GAUGED FLOER HOMOLOGY 27

Conley-Zehnder index of x to be the Conley-Zehnder index of this symplectic path. By the second and ﬁfth axioms listed above, this index induces a well-deﬁned function

CZ : CritAH → Z (4.35) which satisﬁes (2) and (3) of Theorem 4.4. Now we prove (1). For any contractible periodic orbits x : S1 → M of Y and any extension Ht w : D → M of x, we can lift the pair (x, w) to a tuple x = (x, f, [w]) ∈ CritAeH .

Proposition 4.5. If ι : CritAH → CritAH is the inclusion we described in Proposition 2.8, then CZ ◦ ι = CZ. (4.36)

Proof. Since the Conley-Zehnder index is homotopy invariant, and the space of G-invariant ω- compatible almost complex structures is connected, we will compute the Conley-Zehnder index using a special type of almost complex structures, and modify the Hamiltonian H. Starting with any almost complex structure J on M and a G-connection on µ−1(0) → M, J lifts to the horizontal distribution deﬁned by the connection. On the other hand, the biinvariant metric on g gives an identiﬁcation g ' g∗. We denote by η∗ ∈ g∗ the metric dual of η ∈ g. Recall that we have a symplectomorphism

−1 ∗ −1 ∗ µ (g ) ' µ (0) × g . (4.37) ∗ ∗ −1 ∗ For η ∈ g, we define JXη = η ∈ g , as a vector field on µ (0) × g . Then this gives a G-invariant almost complex structure on TM|µ−1(0), compatible with ω. Then we pullback J by the projection −1 ∗ −1 µ (0) × g → µ (0) and denote the pullback by J. −1 ∗ We also modify Ht by requiring that Ht(x, η) = Ht(x) for (x, η) ∈ µ (0)×g . Then the modified Ht can be continuously deformed to the original one, and it doesn’t change H hence doesn’t change

CritAH . Moreover, it is easy to check that for the modiﬁed pair (J, H), h i L J X = L ,JX = 0. (4.38) YHt η YHt η

−1 Now for any (x, w) ∈ CritAH , we lift it to (x, f, [w]) ∈ CritAeH with w : D → µ (0) and f ∗ 1 being a constant θ ∈ g. Then any symplectic trivialization of xe T M → S induces a symplectic trivialization

∗ 1 h 2n−2k i φ : x TM ' S × R ⊕ (g ⊕ g) (4.39) such that φ(Xη,JXζ ) = (0, η, ζ). Then we see, with respect to φ, the operator (4.17) restricted to g4 is         η1 η1 φ − [θ, η2] η1  ψ  d  ψ   η − [θ, φ]  d  ψ       2      7→ Je   +   =: Je + S   . (4.40)  η2  dt  η2   ψ + [θ, η1]  dt  η2  φ φ η1 + [θ, ψ] φ Here we used the property (4.38) and " # " # 0 −Idg⊕g 0 Idg⊕g − adθ Je := ,S = (4.41) Idg⊕g 0 Idg⊕g + adθ 0 28 GUANGBO XU

Moreover, the operator (4.17) respect the decomposition in (4.39). Hence by the fourth axiom of Conley-Zehnder indices we listed above, we have CZ (x, θ, [w]) = CZ(x, w) + CZ eJSte . (4.42) As we have shown in the proof of Proposition 4.1 that for any θ the operator (4.17) is an isomorphism, we can deform θ to zero and compute instead CZ(eJSe 0t) for " # 0 Idg⊕g S0 = , (4.43) Idg⊕g 0 thanks to the homotopy invariance property. Then we see that " # e−t 0 eJSe 0t = (4.44) 0 et which has no eigenvalue on the unit circle for t > 0. Therefore by the third axiom, CZ(eJSe 0t) = 0.

5. Compactness of the moduli space For a general Hamiltonian G-manifold, the failure of compactness of the moduli space of connecting orbits comes from two phenomenon. The ﬁrst is the breaking of connecting orbits, which is essentially the same thing happened in ﬁnite dimensional Morse-Smale-Witten theory. The second is the blow-up of the energy density, which results in sphere bubbling. Since here we have assumed that there exists no nontrivial holomorphic sphere in M, so we only have to consider the breakings.

5.1. Moduli space of stable connecting orbits and its topology. Let’s ﬁx a pair [x±] ∈ CritAH . Denote by Mc([x−], [x+]) := Mc([x−], [x+]; J, H) = M([x−], [x+]; J, H)/R the quotient of the moduli space by the translation in the s-direction. We denote by {ue} the R-orbit in Mc([x−], [x+]) of [ue] ∈ M([x−], [x+]; J, H) and call it a trajectory from [x−] to [x+].

Deﬁnition 5.1. A broken trajectory from [x−] to [x+] is a collection n o n o u := u(α) := u(α), Φ(α), Ψ(α) (5.1) e e α=1,...,m α=1,...,m (α) (α) where for each α, ue ∈ Mc([xα−1] , [xα]) and E(ue ) 6= 0. Here {[xα]}α=0,...,m is a sequence of critical points of AH and [x0] = [x−] , [xm] = [x+] . We regard the domain of eu as the disjoint union m ∪α=1Θ (α) m and let Θ ⊂ ∪α=1Θ the α-th cylinder. We denote by

M ([x−] , [x+]) (5.2) the space of all broken trajectories from [x−] to [x+]. Then naturally we have inclusion

Mc([x−] , [x+]) → M ([x−] , [x+]) . (5.3) GAUGED FLOER HOMOLOGY 29

Deﬁnition 5.2. We say that a sequence of trajectories {uei} = {ui, Φi, Ψi} ∈ Mc([x−] , [x+]) from [x−] to [x+] converges to a broken trajectory n o u := u(α) e e α=1,...,m (1) (2) (m) if: for each i, there exists sequences of numbers si < si < ··· < si and gauge transformations (α) gi ∈ GΘ such that for each α (α)∗ (α)∗ gi si (ui, Φi, Ψi) (5.4)

(α) (α) (α) converges to u , Φ , Ψ on any compact subset of Θ and such that for any sequence of (si, ti) with (α) lim |si − s | = ∞, ∀α i→∞ i we have

lim e (ui)(si, ti) = 0. i→∞ e It is easy to see that this convergence is well-deﬁned and independent of the choices of representatives of the trajectories. We can also extend this notion to sequences of broken connecting orbits. We omit that for simplicity. The main theorem of this section is Theorem 5.3 (Compactness of the moduli space of stable connecting orbits). The space

M ([x−] , [x+]) is a compact Hausdorff space with respect to the topology defined in Definition 5.2. Indeed the proof is routine and it has been carried out in many literature for general symplectic vortex equations, for example [25], [2], [26], [37]. Since bubbling is ruled out, the proof is almost the same as that for finite dimensional Morse theory, while the gauge symmetry is the only additional ingredient.

5.2. Local compactness with uniform bounded energy density. For any compact subset K ⊂ Θ, consider a sequence of solutions uei := (ui, Φi, Ψi) such that the image of ui is contained in the compact subset KH ⊂ M and such that

lim sup E(uei) < ∞. i→∞ We have Proposition 5.4. There exists a subsequence (still indexed by i), a sequence of smooth gauge transformation gi : K → G and a solution ue∞ = (u∞, Φ∞, Ψ∞): K → M × g × g to (2.44) on K, ∗ such that the sequence gi uei converges to ue∞ uniformly with all derivatives on K.

Proof. By the fact that ui is contained in the compact subset KH , and the assumption that there exists no nontrivial holomorphic spheres in M, we have sup e (z) < ∞. (5.5) uei z∈K,i Then this proposition can be proved in the standard way, as did in [2] or [26], using Uhlenbeck’s compactness theorem. 30 GUANGBO XU

5.3. Energy quantization. To prove the compactness of the moduli space, we need the following energy quantization property.

b Proposition 5.5. There exists 0 := 0(J, H) > 0, such that for any connecting orbit ue ∈ MfΘ, we have E(ue) ≥ 0. Proof. Suppose it is not true. Then there exists a sequence of connecting orbits, represented by b solutions in temporal gauge vei := (vi, 0, Ψi) ∈ MfΘ, such that

E(vi) > 0, lim E(vi) = 0. (5.6) e i→∞ e

We ﬁrst know that there is a compact subset KH ⊂ M such that for every i, the image vi(Θ) is contained in KH . Then we must have

lim sup (|∂svi| + |µ(vi)|) = 0. (5.7) i→∞ Θ Indeed, if the equality doesn’t hold, then we can find a subsequence which either bubbles off a nonconstant holomorphic sphere at some point z ∈ Θ (if the above limit is ∞), or (after a sequence of proper translation in s-direction) converges to a solution (with positive energy) on compact subsets (if the above limit is positive and finite). Either case contradicts the assumption. Therefore −1 ∗ we conclude that for any > 0, the image of vi lies in U := µ (g ) for i sufficiently large. −1 Recall that we have projections πµ : U → µ (0) and πµ : U → M. Then for all large i and 0 any s, πµ(vi(s, ·)) is C -close to a periodic orbit of H in M. Since those orbits are discrete (in 0 C -topology, for example), we may fix one such orbit γ ∈ ZeroBH and choose a subsequence (still indexed by i) such that

lim sup sup d (πµ(vi(s, t)), γ(t)) = 0. (5.8) i→∞ (s,t)∈Θ

Then, use a ﬁxed Riemannian metric on M with its exponential map exp, we can write

πµ(vi(s, t)) = expγ(t)ξi(s, t)

1 ∗ ∗ ∗ −1 where ξi ∈ Γ S , γ T M . Let B(γ T M) be the -disk of γ T M. Then expγ pulls back µ (0) → ∗ −1 M to a G-bundle Q → B(γ T M), together with a bundle map γe : Q → µ (0). We can trivialize Q by some ∗ φ : Q → G × B(γ T M).

Now we take a lift xe := (x, f) ∈ ZeroBeH of γ. Then we can write −1 φ γe (x(t)) = (g0(t), γ(t)). (5.9) On the other hand, we write

−1 φ γe πµ(vi(s, t)) = (gi(s, t), ξi(s, t)). (5.10) −1 Take the gauge transformation gei(s, t) = gi(s, t)g0(t) . Then write 0 0 0 0 ∗ vei := (vi, Φi, Ψi) := gei vei. (5.11) GAUGED FLOER HOMOLOGY 31

Then by the exponential convergence of vi as s → ±∞, we see that ∂sgi(s, t) decays exponentially 0 and hence Φi converges to zero as s → ±∞. On the other hand, we see that

−1 0 φ γe πµ(vi(s, t)) = (g0(t), ξei(s, t)). (5.12) Therefore

0 vei ∈ Mf(x,e xe). (5.13)

0 But it is also easy to see that the homotopy class of vei is trivial. Because the energy of connecting 0 orbits only depends on its homotopy class, we see that the energy of vei, and hence the energy of vei is actually equal to zero, which contradicts with the hypothesis.

5.4. Proof of the compactness theorem. It suﬃces to prove, without essential loss of generality, that for any sequence [uei] ∈ M ([x−], [x+]; J, H) represented by unbroken connecting orbits (ui, Φi, Ψi) ∈ Mf(x−, x+), there exists a convergent subsequence. By the assumption that there exists no nontrivial holomorphic sphere in M, we have

sup |∂sui + XΦi (ui)| < +∞. (5.14) i,Θ

Then the limit (broken) connecting orbits can be constructed by induction and the energy quantization property (Proposition 5.5) guarantees that the induction stops at ﬁnite time. The details are standard and left to the reader.

6. Floer homology

In this section we use the moduli spaces M ([x−], [x+]; J, H) to deﬁne the vortex Floer homology group VHF∗ (M, µ; H). We also discuss further works and related problems in the last three subsections. By Corollary A.12, we can choose a generic S1-family of “admissible” almost complex structures reg J ∈ JeH which is regular with respect to H. Such an object is a smooth t-dependent family of almost complex structures Jt, such that for each t, Jt is G-invariant, ω-compatible, and outside a −1 C neighborhood U of µ (0), Jt ≡ J; inside U, Jt preserves a distribution gU . Being regular implies that for all pairs [x±] ∈ CritAH , the moduli space M ([x−], [x+]; J, H) is a smooth manifold with

dimM ([x−], [x+]; J, H) = CZ([x−]) − CZ([x+]). (6.1)

Moreover, there is free R-action on M ([x−], [x+]; J, H) by time translation, whose orbit space is denoted by Mc([x−], [x+]; J, H). Combining the compactness theorem, we have

reg Proposition 6.1. If J ∈ JeH , then

CZ([x−]) − CZ([x+]) ≤ 0 =⇒ M ([x−], [x+]; J, H) = ∅; (6.2)

CZ([x−]) − CZ([x+]) = 1 =⇒ #Mc([x−], [x+]; J, H) < ∞. (6.3) 32 GUANGBO XU

6.1. The gluing map and coherent orientation. The boundary operator of the Floer chain complex is defined by the (signed) counting of Mc([x−], [x+]; J, H). If we want to define the Floer homology over Z2, then we don’t need to orient the moduli space; otherwise, the orientation of Mc([x−], [x+]; J, H) can be treated in the same way as the usual Hamiltonian Floer theory, since the augmented linearized operator D (whose determinant is canonically isomorphic to the deter- ue minant of the actual linearization dF ), is of the same type of Fredholm operators considered in [ue] the abstract setting of [10]. We first give the gluing construction and then discuss the coherent orientations of the moduli spaces. In this subsection we construct the gluing map for broken trajectories with only one breaking. The general case is similar. This construction is, in principle, the same as the standard construction in various types of Morse-Floer theory (see [32][10]), with a gauge-theoretic flavor. The gauge symmetry makes the construction a bit more complex, since we always glue representatives, and we want to show that the gluing map is independent of the choice of the representatives. reg In this subsection, we fix the choice of the admissible family J ∈ JeH and omit the dependence of the moduli spaces on J and H. For any pair x± ∈ CritAH , we say that a solution ue ∈ Mf(x−, x+) is in r-temporal gauge, if its restrictions to [r, +∞) × S1 and (−∞, r] × S1 are in temporal gauge, for some r > 0. Now we fix a number r = r0 and only consider solutions in r0-temporal gauge. Now we take three elements x, y, z ∈ CritAeH with CZ(x) − 1 = CZ(y) = CZ(z) + 1. (6.4)

1 Assume y = (y, η): S → M × g. We would like to construct, for a large R0 > 0, the gluing map

glue : Mc([x], [y]) × (R0, +∞) × Mc([y], [z]) → Mc([x], [z]) . (6.5)

Now consider two trajectories [ue±] = [u±, Φ±, Ψ±], [ue−] ∈ M ([x], [y]), [ue+] ∈ M ([y], [z]) with their representatives both in r0-temporal gauge and ue± are asymptotic to y as s → ∓∞. Then there exists R1 > 0 such that

±s ≥ R1 =⇒ u∓(s, t) = expy(t) ξ∓(s, t). (6.6) Here Θ+ = [R , +∞) × S1 and Θ− = (−∞, −R ] × S1 and ξ ∈ W k,p Θ∓ , y∗TM . R1 1 R1 1 ± R1 Next, we take a cut-oﬀ function ρ such that s ≥ 1 =⇒ ρ(s) = 1, s ≤ 0 =⇒ ρ(s) = 0. For each

R >> r0, denote ρR(s) = ρ(s − R). We construct the “connected sum”  u (s + R, t), s ≤ − R − 1  − 2  R R uR(s, t) = expy(t) ρ R (−s)ξ−(s + R, t) + ρ R (s)ξ+(s − R, t) s ∈ − − 1, + 1 (6.7) 2 2 2 2  R u+(s − R, t), s ≥ 2 + 1

 (Φ (s + R, t), Ψ (s + R, t)) , s ≤ − R − 1  − − 2  R R (ΦR, ΨR)(s, t) = 0, ρ R (−s)Ψ−(s + R, t) + ρ R (s)Ψ+(s − R, t) , s ∈ − − 1, + 1 (6.8) 2 2 2 2  R (Φ+(s − R, t), Ψ+(s − R, t)) . s ≥ 2 + 1 Now it is easy to see that, if we change the choice of representatives ue± which are also in r0-temporal R R gauge, the connected sum ueR := (uR, ΦR, ΨR) doesn’t change for s ∈ [− 2 − 1, 2 + 1] and hence GAUGED FLOER HOMOLOGY 33 we obtain a gauge equivalent connected sum. Moreover, if we change y by y0 which represent the same [y] ∈ CritAH , then we can obtain 0 0 0 0 ue− ∈ Mf x, y , ue+ ∈ Mf y , z (6.9) 0 which are also in r0-temporal gauge, and we obtain a connected sum ueR which is gauge equivalent to ueR. Now we consider the augmented linearized operator D := D . R ueR 2,p 2,p Lemma 6.2. There exists c > 0 and R0 > 0 such that for every R ≥ R0 and η ∈ Ee ⊕W (Θ, g), e ueR we have ∗ ∗ kDRηekW 1,p ≤ c kDRDRηekLp . (6.10) Proof. Same as the proof of [32, Proposition 3.9] Hence we can construct a right inverse ∗ ∗ −1 0,p p 1,p QR := DR (DRDR) : Ee ⊕ L (Θ, g) → Tu Be (6.11) ueR eR with

kQRk ≤ c. (6.12)

0,p 1,p Now we write QR := (QR, AR) with QR : Ee → Tu Be (x, z). Then actually the image of QR ueR eR ∗ lies in the kernel of d and therefore Q is a right inverse to dF | ∗ . Because our construction 0 R eueR kerd0 is natural with respect to gauge transformations, we see that QR induces an injection 0,p 1,p QR : E → T[u ]B (6.13) [ueR] eR which is a right inverse to the linearized operator dF and which is bounded by c. By the implicit [ueR] function theorem, we have

Proposition 6.3. There exists R1 > 0, δ1 > 0 such that for each R ≥ R1, there exists a unique ∗ 1,p ξe ∈ ImQR = kerd0 ⊂ Tu Be , ξe ≤ δ1 such that eR W 1,p Fe expu ξe = 0, ξe ≤ 2c Fe(uR) . (6.14) eR W 1,p e Lp Therefore, the gluing map can be defined as h i glue ([u−],R, [u+]) = exp ξ ∈ M ([x], [z]; J, H) . (6.15) e e ueR e c On the other hand, it is easy to see that the augmented linearized operators D for all con- ue necting orbits ue is of “class Σ” considered in [10]. Therefore, by the main theorem of [10], there exists a “coherent orientation” with respect to the gluing construction. Choosing such a coherent orientation, then to each zero-dimensional moduli space Mc([x], [y]; J, H), we can associate the counting χJ ([x], [y]) ∈ Z, where each trajectory [ue] contributes to 1 (resp. -1) if the orientation of [ue] coincides (resp. differ from) the “flow orientation” of the solution. Then we define

δJ : VCFk (M, µ; H;ΛZ) → VCFk−1 (M, µ; H;ΛZ) X (6.16) [x] 7→ χJ ([x], [y])[y]

[y]∈CritAH 34 GUANGBO XU

As in the usual Floer theory, we have

Theorem 6.4. For each choice of the coherent orientation on the moduli spaces M([x], [y]; J, H), the operator δJ in (6.16) deﬁnes a morphism of ΛZ-modules satisfying δJ ◦ δJ = 0.

This makes (VCF∗(M, µ; H;ΛZ), δJ ) a chain complex of ΛZ-modules, to which will be generally referred as the vortex Floer chain complex. Therefore the vortex Floer homology group is deﬁned as the ΛZ-module

ker (δJ : VCFk (M, µ; H;ΛZ) → VCFk−1 (M, µ; H;ΛZ)) VHFk (M, µ; J, H;ΛZ) := (6.17) im (δJ : VCFk+1 (M, µ; H;ΛZ) → VCFk (M, µ; H;ΛZ))

6.2. The continuation map. Now we prove that the vortex Floer homology group defined above is independent of the choice of admissible family of almost complex structures and the time-dependent Hamiltonians, and, if we use the moduli space of (1.8) instead of (1.7) to define the Floer homology, independent of the parameter λ > 0. So far the argument has been standard, by using the continuation principle. Let (J α,Hα, λα) and J β,Hβ, λβ be two triples where λα, λβ > 0, Hα,Hβ are G-invariant α reg β reg Hamiltonians satisfy Hypothesis 2.1 and 2.3, and J ∈ JeHα,λα , J ∈ JeHβ ,λβ (for notations refer to the appendix). We choose a cut-off function ρ : R → [0, 1] with s ≤ −1 =⇒ ρ(s) = 1 and s ≥ 1 =⇒ ρ(s) = 0. Then we define

α β α β Hs,t = ρ(s)Ht + (1 − ρ(s))Ht , λs = ρ(s)λ + (1 − ρ(s))λ . (6.18)

We denote this family of Hamiltonians by H . Now, as in the appendix, we consider the space of α β families of admissible almost complex structures Je J ,J consisting of smooth families of almost complex structures J = (Js,t)(s,t)∈Θ, such that for each l ≥ 1,

|s| α |s| β e Js,t − J < ∞, e Js,t − J < ∞. (6.19) t l t l C (Θ−×M) C (Θ+×M)

α β This is a Fr´echet manifold. For any J ∈ Je J ,J , we consider the following equation on ue = (u, Φ, Ψ)  ∂u ∂u  + X (u) + J + X (u) − Y (u) = 0;  Φ s,t Ψ Hs,t ∂s ∂t (6.20) ∂Ψ ∂Φ  − + [Φ, Ψ] + λ2µ(u) = 0.  ∂s ∂t s For the same reason as in Section3, any ﬁnite energy solution whose image in M has compact α β closure is gauge equivalent to a solution which is asymptotic to x ∈ CritAeHα (resp. x ∈ CritAeHβ ) α β as s → −∞ (resp. s → +∞). Hence for any [x ] ∈ CritAHα and [x ] ∈ CritAHβ , we can consider the moduli space of solutions

α β N [x ], [x ]; J , H , λs . (6.21) GAUGED FLOER HOMOLOGY 35

One thing to check in deﬁning the continuation map is the energy bound of solutions, which implies the compactness of the moduli space. We deﬁne the energy to be

Proposition 6.5. For any solution ue to (6.20) whose energy is ﬁnite and whose image in M has compact closure, we have Z α β ∂Hs,t E (ue) = AHα ([x ]) − AHβ ([x ]) − (u)dsdt. (6.23) Θ ∂s

Proof. We can transform the solution ue into temporal gauge. Then the energy density is 2 2 ∂Ψ |∂su| + |λsµ(u)| =ω ∂su, ∂tu + XΨ − YHs,t (u) − µ(u) · ∂s (6.24) ∂ ∂ ∂H =ω(∂ u, ∂ u) − (µ(u) · Ψ) + (H (u)) − s,t (u). s t ∂s ∂s s,t ∂s Then integrating over Θ, we obtain (6.23). Theorem 6.6. There exists a Baire subset J reg J α,J β ⊂ J J α,J β, such that for any ∈ eH ,λs e J J reg J α,J β, the moduli space N ([xα], [xβ]; , , λ ) is a smooth oriented manifold with eH ,λs J H s α β α β dimN [x ], [x ]; J , H , λs = CZ([x ]) − CZ([x ]). (6.25)

So in particular, when CZ(xα) = CZ(xβ), N is of zero dimension. The algebraic count of N gives an integer χ [xα], [xβ]. Then we deﬁne the continuation map

β α α α β β β contα : VCF∗ (M, µ; J ,H , λ ;ΛZ) → VCF∗ M, µ; J ,H , λ ;ΛZ X [xα] 7→ χ [xα], [xβ] [xβ]. (6.26) β [x ]∈CritAHβ Now we have the similar results as in ordinary Hamiltonian Floer theory.

β Theorem 6.7. The map contα is a chain map. The induced map on the vortex Floer homology groups is independent of the choice of the homotopy H , the family J of almost complex structures, β γ γ the cut-oﬀ function ρ. In particular, contα is a chain homotopy equivalence. If (J ,H ) is another admissible pair and λγ > 0, then in the level of homology

γ β γ contβ ◦ contα = contα. Proof. The proof is essentially based on the construction of various gluing maps and the compact- α β α β ness results about N [x ], [x ]; J , H , λs when CZ([x ]) − CZ([x ]) = 1. As in the gluing map 2 constructed in proving the property δJ = 0, we need to specify a gauge to construct the approxi- mate solutions. We can still use solutions in r-temporal gauge, which is a notion independent of the equation. We omit the details. 36 GUANGBO XU

6.3. Computation and Morse homology. In this subsection we discuss the computation of the vortex Floer homology group. Before we proceed let us recall how to show that the ordinary Hamiltonian Floer homology is isomorphic to the Morse homology. On a compact symplectic manifold M we take the Hamiltonian to be the t-independent function f, where is small and f is a Morse function on the manifold M. Then periodic orbits of f corresponds to critical points of f (which are denoted by z1, . . . , zk), and the Conley-Zehnder index and the Morse index are related by

CZ(zi) = n − Ind(zi) where 2n = dimM. Then the Floer chain complex is generated by (zi, wi), where wi is a spherical class. Then we want to show that when

CZ(zi, wi) − CZ(zj, wj) = 0, 1 (6.27) by taking a generic t-independent almost complex structure J on M, all Floer trajectories connecting (zi, wi) and (zj, wj) are t-independent, i.e., corresponds to Morse-Smale trajectories of f with respect to the metric ω(·,J·). Hence the boundary operators in the Floer chain complex and the Morse-Smale-Witten chain complex coincide. So that

2n−∗ HF∗(M, f; Λ) ' HM (M, f; Z) ⊗ Λ (6.28) The main diﬃculty to carry out the above argument is, when J is t-independent, one cannot easily achieve the transversality of the moduli space of Floer trajectories. Here the cylinder Θ may have a ﬁnite cover over itself

πk(s, t) = (ks, kt) (6.29) and there might exist Floer trajectories which are multiple covers of other trajectories (when J is allowed to vary with t, such objects don’t exist generically). The multiple covers might have higher dimensional moduli than expected, which is similar to the problem caused by the negatively covered spheres in Gromov-Witten theory. Hence to overcome this difficulty, one has to either put topological restrictions (such as semi-positivity in [28], [19]), or to use virtual technique, to say that, though the negative multiple covers have higher dimensional moduli, they contributes to zero in defining the boundary operator (see [15], [23]) Now back to the case of vortex Floer homology. We choose a Morse function f : M → R so that the induced Hamiltonian Ht := f has its periodic orbits corresponding to the critical points of f. Then we lift f to a function f : M → R and consider the vortex Floer homology defined for the

Hamiltonian f. To show that the resulting homology group is isomorphic to H∗(M;ΛQ), we have to show that for small enough, the counting of Floer trajectories corresponds to the counting of negative gradient ﬂow trajectories of a Lagrange multiplier function associated to f. If we choose to put topological restrictions to avoid virtual technique, then one diﬃculty is the 1 following. To achieve transversality, we assumed that Ht vanishes for certain t ∈ S in the appendix. This condition doesn’t hold for f, which is independent of t. And this time the transversality much be achieved by only using a t-independent almost complex structure J. So virtual technique is probably unavoidable in this approach. GAUGED FLOER HOMOLOGY 37

In ordinary Hamiltonian Floer homology, another way to prove the isomorphism is the so-called Piunikhin-Salamon-Schwarz (PSS) construction, introduced in [30]. It is to consider the moduli space of “spiked disks”, which is an object interpolating between Floer trajectories and Morse trajectories. The counting of spiked disks deﬁnes a pair of chain maps between the Floer chain complex and the Morse-Smale-Witten chain complex, and using various gluing/stretching constructions one can prove that the two chain maps are homotopy inverses to each other. In the second paper of this series, we will give a PSS type construction to prove the isomorphism between VHF∗(M, µ;ΛZ) and the Morse homology of the symplectic quotient M.

Appendix A. Transversality by perturbing the almost complex structure In this appendix, we treat the transversality of our moduli space of connecting orbits. For general symplectic manifold, connecting orbits may develop sphere bubbles, while the expected dimension of the moduli space of such sphere bubbles may be even larger than the expected dimension of the moduli space of connecting orbits. In this case one must use the virtual technique to say something of the structure of the compactified moduli space of connecting orbits. The boundary operator is defined by the virtual count of the number of trajectories, therefore the Floer homology is only defined over Q in general. Instead, in this section we restrict to the case where the virtual technique is not necessary. We remark that this special case covers most interesting examples to which we will apply our results (for example, toric manifolds as symplectic quotient of vector spaces).

First we recall the important assumption on the Hamiltonian Ht.

1 Hypothesis A.1. There exists a nonempty open subset I ⊂ S such that Ht(x) = 0 for t ∈ I.

Indeed, for any G-invariant Hamiltonian diﬀeomorphism of M, if it is given by the time-1 map of some Hamiltonian path, then we can reparametrize the Hamiltonian path to make it vanish for t lying in a small interval, while the time-1 map of the reparametrized path is the original Hamiltonian diﬀeomorphism. Hence this hypothesis is not an essential restriction.

A.1. Admissible family of almost complex structures. We know that for any > 0 small −1 ∗ −1 ∗ enough, there exists a symplectomorphism U = U := µ (g ) ' µ (0) × g . Hence we have a ∗ natural projection πµ : U → M. There is a natural foliation on U, whose leaves are Gx × g with x ∈ µ−1(0), with dimension equal to 2dimG. The tangent planes of this foliation is a G-invariant C distribution on U, denoted by gU . Recall that we are also given an almost complex structure J in Hypothesis 2.4. Now we will perturb J in a speciﬁc way. This approach was similar to that in Woodward’s erratum for [36], but here we don’t have a Lagrangian submanifold.

Definition A.2. An admissible almost complex structure on M is a G-invariant, ω-compatible C almost complex structure J which preserves the distribution gU on U, and coincides with J outside U. The set of all admissible almost complex structures is denoted by J (M, U, J). We denote by Je(M, U, J) the space of smooth S1-families of admissible almost complex structures, and define J l(M, U, J) and Jel(M, U, J) the corresponding objects in the Cl-category, for l ≥ 1. We abbreviate them by J l, Jel because M, U, J are all fixed. 38 GUANGBO XU

ω and any J ∈ J l induces a G-invariant Riemannian metric g on M. We denote by T J the J M orthogonal complement of gC with respect to the metric g . Then T J is isomorphic to π∗ T M, and U J M µ we have the orthogonal splitting:

∗ C J C TU ' πµT M ⊕ gU =: TM ⊕ gU . (A.1) C l Now by the integrability of gU , we see that for any a, b ∈ g and any J ∈ J , we have C [JXa,JXb] ∈ gU . (A.2)

l l Lemma A.3. For l ≥ 1, the space Je is a smooth Banach manifold. For any J = {Jt}t∈S1 ∈ Je , l 1 the tangent space TJ Je is naturally identiﬁed with the space of G-invariant sections E : S × M → l 1 EndRTM (of class C ), supported in the closure of U, and for each t ∈ S satisfying i. JtEt + EtJt = 0; ii. ω(·,Et·) is a symmetric tensor; C iii. gU is invariant under Et. k,p Now we consider the following equation for ue := (u, Φ, Ψ) ∈ Wloc (Θ,M × g × g) with Ht satisfying Hypothesis (A.1) and J ∈ Jel:   ∂su + XΦ(u) + Jt (∂tu + XΨ(u) − YHt (u)) = 0; (A.3)  ∂sΨ − ∂tΦ + [Φ, Ψ] + µ(u) = 0.

We consider only ﬁnite energy solutions and for any pair x± ∈ CritAeH , denote by Mf(x±; J, H) the space of all solutions which are asymptotic to x±. k,p 2 We can also identify any solution ue with an object ve := (v, Φ, Ψ) ∈ Wloc R ,M × g × g by H v(s, t) = ϕt (u(s, t)), and Φ, Ψ lifts periodically in t ∈ R. It satisﬁes  H  ∂sv + XΦ(v) + Jt (∂tv + XΨ(v)) = 0; (A.4)  ∂sΨ − ∂tΦ + [Φ, Ψ] + µ(u) = 0;

H H Here Jt = φt ∗ Jt for t ∈ R. For Cl almost complex structures, we have the following regularity theorem:

1,p Theorem A.4. [2, Theorem 3.1] For any l ≥ 1 and any solution ue ∈ Wloc (Θ,M × g × g) to (A.3) l 2,p ∗ l+1,p with J ∈ Je , there exists a gauge transformation g ∈ Gloc (Θ,G) such that g ue ∈ Wloc (Θ,M × g × g).

A.2. Existence of injective points. In this subsection we prove an important technical result, showing that for any admissible family of complex structures J ∈ Jel and any nontrivial connecting orbit, there exist “injective” points and they form a rather large subset of Θ. We fix l ≥ 1 and J in this subsection. We first generalize the notion of injective points in [11]. For any C1-map u :Θ → X with lims→±∞ u(s, t) = x±(t), we define U −1 U U Θ (u) := u (U), ΘI (u) := Θ (u) ∩ (R × I); (A.5) n o U C CI (u) = (s, t) ∈ ΘI (u) | ∂su(s, t) ∈ gU ; (A.6) GAUGED FLOER HOMOLOGY 39

U RI (u) = (s, t) ∈ ΘI (u) \ C (ue) | u(s, t) ∈/ Gu ((R \{s}) × {t}) , u(s, t) ∈/ Gx±(t) . (A.7) Note that the above sets are unchanged if we apply to u a gauge transformation.

Lemma A.5. For any nontrivial connecting orbit ue = (u, Φ, Ψ) ∈ Mf(x±; J, H), the set CI (u) is U discrete in ΘI (u).

Proof. We can assume that ue is in temporal gauge, i.e., Φ ≡ 0. Denote ξ = ∂su. The vector ξe = (ξ, 0, ∂sΨ) lies in the kernel of the linearized operator. So in particular, over R × I,

∇sξ + (∇ξJt)(∂tu + XΨ) + Jt (∇tξ + ∇ξXΨ) = −JtX∂sΨ. (A.8)

The family Jt induces a family of metrics on M, and hence a decomposition u∗TM ' u∗T Jt ⊕ gC (A.9) M U U U over Θ (u). Suppose ξ = ξ + Xa + JtXb for two functions a, b :Θ (u) → g and ξ(t) ∈ √ Γ ΘU (u), u∗T Jt . Denote σ = a + −1b ∈ gC and X + J X = X . Then over ΘU (u), M a t b σ I

C ∇sξ =∇sξ + X∂sσ + ∇ξXσ + ∇Xσ Xσ ≡ ∇sξ + ∇ξXσ + ∇Xσ Xσ mod gU ;

(∇ξJt)(Jtξ) =(∇ξJt)(Jtξ) + (∇Xσ Jt)(Jtξ) + (∇Xσ Jt)(JtXσ); (A.10) J (∇ ξ + ∇ X ) =J (∂ J )X + ∇ ξ + X + ∇ X + ∇ X + ∇ X t t ξ Ψ t t t b t ∂tσ Jtξ+JtXσ−XΨ σ ξ Ψ Xσ Ψ C ≡Jt∇tξ + Jt∇JtXσ Xσ + Jt∇JξXσ + Jt∇ξXΨ mod gU .

C (Note that (∂tJt)Xb ∈ gU .) Therefore we have

0 ≡ ∇sξ + Jt∇tξ + ∇Xσ Xσ + Jt∇JtXσ Xσ + (∇Xσ Jt)(JtXσ) + ∇ X + (∇ J )(J ξ) + (∇ J )J ξ + J ∇ X + J ∇ X ξ σ ξ t t Xσ t t t Jtξ σ t ξ Ψ (A.11) 0,1 = ∇ ξ + C(z)ξ + Jt[JtXσ,Xσ]

0,1 C ≡ ∇ ξ + C(z)ξ mod gU . Here ∇ is the connection on u∗T Jt induced from the Levi-Civita connection for g = ω(·,J ·) by M t t orthogonal projection, and C(z) is a zero-order operator and the last congruence follows from the C integrability of gU . Hence the section ξ satisﬁes the Cauchy-Riemann equation 0,1 ∇ ξ + C(z)ξ = 0. (A.12)

Here by ∇ is the connection on u∗T Jt induced from ∇ by the splitting (A.1). M Now we apply the Carleman similarity principle of [11]. We see that the vanishing set of ξ is U discrete or ξ ≡ 0 on ΘI (u). If the latter happens, then it is easy to see that ue is a trivial connecting orbit. 1,p Lemma A.6. Suppose we have (ui, Φi, Ψi) ∈ Wloc (Θ,M × g × g), i = 1, 2, satisfying

∂sui + XΦi (ui) + Jt (∂tui + XΨi (ui)) = 0 (A.13) 2,p on an open subset V ⊂ Θ with ui(V ) ⊂ U, such that there exists g ∈ Wloc (V,G) with g(z)u2(z) = ∗ u1(z) for z ∈ V . Then g (u1, Φ1, Ψ1) = (u2, Φ2, Ψ2) on V . 40 GUANGBO XU

Proof. We assume that g ≡ 1. Then u1 ≡ u2 ≡ u. By the equation

∂su + XΦi (u) + Jt (∂tu + XΨi − YHt ) = 0, (A.14)

we see that XΦ1 (u) = XΦ2 (u), XΨ1 (u) = XΨ2 (u). Since u(V ) ⊂ U we see that Φ1 ≡ Φ2,Ψ1 ≡ Ψ2.

1,p Proposition A.7. Let uei = (ui, Φi, Ψi) ∈ Mf (xe±; J, H), i = 1, 2 be solutions to (A.3) on Θ 1 for some J ∈ Je , such that they coincide on a small disk B ⊂ Θ. Then there exists a gauge 2,p ∗ transformation g ∈ G such that g ue2 = ue1.

Proof. We apply the approach in [5, Section 4.3.4] using the unique continuation results of [27]. 2 I. We identify uei with two solutions vei := (vi, Φi, Ψi) to (A.4) over R . By deﬁnition of vei and 2 the hypothesis, ve1 and ve2 coincide on a small disk in R . We will ﬁrst prove that for all z0 ∈ Θ, there exists an open disk Bρ(z0) centered at z0 on which ve1 and ve2 are gauge equivalent. In fact, 2 2 all such points in R form a nonempty open subset Ω. If Ω 6= R , choose the largest ρ0 > 0 such that Bρ0 = Bρ0 (0) ⊂ Ω. Then we can patch gauge transformations together to obtain a global gauge transformation g : 2 → G such that g∗v = v over the closure of B (since B is simply R e1 e2 ρ0 ρ0 connected). So without loss of generality, we assume that ve1 and ve2 coincide over the closure of

Bρ0 . 2 Then we want to show that indeed every z1 ∈ ∂Bρ0 also lies in Ω. Take a small disk Bρ1 (z1) ⊂ R centered at z1 such that both v1 and v2 map Bρ1 (z1) into a coordinate chart V ⊂ M, and with n n respect this coordinate chart, the almost complex structure Jz(vi(z)) : C → C satisﬁes

1 kId + J (v (z))I k ≤ (A.15) z i 0 2

n where I0 is the standard complex structure on C .

Then we can ﬁnd z2 ∈ Bρ0 which is on the line segment between 0 and z1 satisfying the following conditions.

(1) There exists ρ2 > 0 such that z1 ∈ Bρ2 (z2) ⊂ Bρ1 (z1);

(2) The gauge transformations gi : Bρ2 (z2) → G deﬁned by parallel transport along the radial direction with respect to the connection Ai satisﬁes

0 −1 0 gi(w ) vi(w ) ∈ V. (A.16)

(This is because ρ2 is so small so gi is very close to the identity of G. Now we will show that v and v are gauge equivalent over B (z ). We regard v and v are e1 e2 ρ2 2 1 2 n maps into V ⊂ C and by (A.16), we may assume that the connections are in radial gauge, i.e., the connection forms are fidθ, i = 1, 2. Then Xfi is a vector ﬁeld on V . Then we have

∂v ∂v i + J (v (z)) i + X (v (z)) = 0. (A.17) ∂r z i ∂θ fi(z) i GAUGED FLOER HOMOLOGY 41

Subtract one from another (using the linear structure of V ), denoting ξ(z) = v2(z) − v1(z) and h(z) = f2(z) − f1(z), we obtain ∂ξ ∂ξ + J (v (z)) ∂r z 1 ∂θ ∂v = − (J (v ) − J (v )) 2 − J (v )X (v ) + J (v )X (v ) z 2 z 1 ∂θ z 2 f2 2 z 1 f1 1 ∂v = (J (v ) − J (v )) 2 − J (v )X (v ) − (J (v ) − J (v ))X − J (v )(X (v ) − X (v )) z 1 z 2 ∂θ z 1 h 1 z 2 z 1 f2 z 2 f2 2 f2 1 = : R1(v1, v2, f1, f2). (A.18) Similarly, the diﬀerence between the vortex equations in the radial gauge gives ∂h = −r(µ(v ) − µ(v )) =: R (v , v , f , f ). (A.19) ∂r 2 1 2 1 2 1 2 It is easy to see that there exists a constant K > 0 such that

|Rj(v1, v2, f1, f2)(r, θ)| ≤ K (|ξ| + |h|) , j = 1, 2. (A.20) Hence we have a diﬀerential inequality ! ! ∂ ξ J (v ) ∂ξ z 1 ∂θ + ≤ K |(ξ, h)| . (A.21) ∂r h 0

We replace ξ by ζ(z) := (Id − I0Jz(v1(z)))ξ(z) and obtain ∂ζ ∂ζ + I ∂r 0 ∂θ ∂Jz(v1) ∂Jz(v1) ∂ξ ∂ξ = − I0 − ξ + (Id − Jz(v1)I0) + (I0 + Jz(v1)) (A.22) ∂r ∂θ ∂r ∂θ ∂J (v ) ∂J (v ) ∂ξ ∂ξ = − I z 1 − z 1 ξ + (Id − J (v )I ) + J (v ) . 0 ∂r ∂θ z 1 0 ∂r z 1 ∂θ

∂ζ 1 If we denote by P (ζ, h) = I0 ∂θ , 0 , then P is an self-adjoint diﬀerential operator on the circle S . Then we have the diﬀerential inequality for some K0 > 0

d 0 (ζ, h) + P (ξ, h) ≤ K k(ζ, h)kL2(S1) . (A.23) dr L2(S1)

This is in the form considered in [27]. Since (ζ, h) vanishes for r small, (ζ, h) ≡ 0 for r ≤ ρ2.

This implies that in radial gauge, the two solutions coincide on Bρ2 (z2), which contains z1. Hence

∂Bρ0 ⊂ Ω. By the compactness of ∂Bρ0 , we obtain a disk strictly larger than Bρ0 which is also 2 contained in Ω. This contradicts with the deﬁnition of ρ0. Therefore Ω = R . ∗ II. Now we prove that there exists a global gauge transformation g :Θ → G such that g ue2 = ue1. Indeed, there exists S > 0 such that

1 U U (−∞, −S] × S ⊂ Θ (u1) ∩ Θ (u2). (A.24) By the property of U, we see that the local gauge transformations around each point z ∈ (−∞, −S]× R appeared in I. are unique. Hence we can patch them to obtain a global gauge transformation 1 ∗ g :(−∞, −S] × S → G such that g ue2 = ue1. But it is easy to see that we can extend g to a longer 42 GUANGBO XU

1 cylinder (−∞, −S + 0] × S with 0 independent of S (we omit the details). Hence there exists a global gauge transformation which identiﬁes ue1 with ue2. 2,p Lemma A.8. Suppose we have two solutions (ui, Φi, Ψi) ∈ W (Bri ,M × g × g) on Bri ⊂ R × I, i = 1, 2 to (A.3), for some J ∈ Jel, l ≥ 2, satisfying the following conditions:

(1) ui(Bri ) ⊂ U, u1(0) = u2(0); C (2) ui(Bri ) is an embedded surface which intersects each leaf of gU cleanly at at most one point; 0 0 (3) For each (s, t) ∈ Br1 , there exists g ∈ G and (s , t) ∈ Br2 such that u1(s, t) = gu2(s , t). ∗ Then r1 ≤ r2 and there exists a gauge transformation g : Br1 → G such that g (u1, Φ1, Ψ1) =

(u2, Φ2, Ψ2) over Br1 . Proof. We can ﬁnd an embedded submanifold S ⊂ U of codimension dimG, transverse to G-orbits, such that u2(Br2 ) ⊂ S. Then we can construct a smooth map π : G · S → Br2 such that π ◦ v = Id.

The hypothesis implies that u1(Br1 ) ⊂ Gu2(Br2 ). Then the map π ◦ u1 must take the form (s, t) 7→ (φ(s, t), t), and there exists a unique g(s, t) ∈ G such that

u1(s, t) = g(s, t)u2(φ(s, t), t). (A.25) Now since u is a solution to the vortex equation, we see on B , ei r1

0 = ∂su1 + XΦ1 (u1) + Jt(∂tu1 + XΨ1 (u1)) ≡ (∂su2) ∂sφ + Jt (∂tu2 + (∂su1) ∂tφ) (A.26) C ≡ (∂su2)(∂sφ − 1) + (∂tu2) ∂tφ mod gU .

C But since u2 intersects with leaves of gU cleanly, we see that the above congruence holds if and only if ∂sφ ≡ 1 and ∂tφ ≡ 0. Hence there exists s0 ∈ R such that u1(s, t) = g(s, t)u2(s + s0, t) for each (s, t) ∈ Br1 . Since u1(0, 0) = u2(0, 0) and the second hypothesis, s0 = 0. Hence r1 ≤ r2 and by Lemma A.6, the restriction of u to B is gauge equivalent to u . e2 r1 e1 l Proposition A.9. For any J ∈ Je (l ≥ 2), and any ue = (u, Φ, Ψ) ∈ Mf(x±; J, H) a nontrivial U connecting orbit, the set RI (u) is open and dense in ΘI (u).

Proof. I. We first prove that RI (u) is open. If it is not the case, then there exists (s0, t0) ∈ RI (u) U and a sequence (si, ti) ∈ ΘI (u) \ RI (u) such that limi→+∞(si, ti) = (s0, t0). By the definition of 0 RI (u), we must have that for each i large enough, there exists gi ∈ G and si ∈ R \{si} such that 0 u(si, ti) = giu(si, ti). (A.27) 0 0 If si is unbounded, then we have that (for a subsequence) u(si, ti) → x±(t0) as i → +∞. This 0 contradicts with the condition that u(s0, t0) ∈/ Gx±(t0). Hence si must be bounded. So we may 0 0 0 assume that si → s0 ∈ R. This implies that u(s0, t0) = g0u(s0, t0) for some g0 ∈ G. By the 0 0 definition of RI (u), we must have s0 = s0. Hence the two different sequences (si, ti) and (si, ti) C both converges to (s0, t0). Then (A.27) implies that ∂su(s0, t0) ∈ gU ⊂ gU , which contradicts with the fact that (s0, t0) ∈ RI (u). U II. Now we prove that RI (u) is dense. Since we know that the subset CI (u) is discrete in ΘI (u), U we only have to show that every point in ΘI (u) \ CI (u) can be approximated by points in RI (u). U We take a point (s0, t0) ∈ ΘI (u) \ CI (u). We may also assume that u(s0, t0) ∈/ Gx±(t0); otherwise, for any other s close to s0,(s, t0) satisfies this condition. GAUGED FLOER HOMOLOGY 43

Now we assume that (s0, t0) is not in the closure of RI (u). Then there exists 0 > 0 and a nonempty open subset I0 ⊂ I with U B0 (s0, t0) ⊂ ΘI0 (u) \ RI0 (u). (A.28)

We may choose 0 small enough and S large enough so that

(1) For any |s| ≥ S, |t − t0| ≤ 0, u(s, t) ∈/ Gu (B0 (s0, t0)); (2) For |t − t0| ≤ 0, u ([s0 − 0, s0 + 0] × {t}) is embedded and intersects with each G-orbit at at most one point. (3) Gu (B0 (s0, t0)) ∩ G CI (u) ∩ [−S, S] × I0 = ∅. 0 Now the condition (A.28) implies that for all (s, t) ∈ B0 (s0, t0), there exists s ∈ R \{s} such that u(s, t) ∈ Gu(s0, t). II.a) We claim that for each (s, t), there exists only finitely many such s0. Indeed, the first condition above implies that, if there are infinitely many such s0, then they must have an accumulation C point in [−S, S], and at the accumulation point ∂su lies in gU . This contradicts with the third condition. So the claim is true.

II. b) For any (s, t) ∈ B0 (s0, t0), deﬁne KS((s, t)) := (s0, t) ∈ [−S, S] × I | s0 6= s, u(s, t) ∈ Gu(s0, t) (A.29) and N := min #KS((s, t)) ≥ 1. (A.30) (s,t)∈B0 (s0,t0) ∗ ∗ ∗ ∗ We claim that there exists (s , t ) ∈ B0 (s0, t0) and 1 > 0 such that B1 (s , t ) ⊂ B0 (s0, t0), and ∗ ∗ S (s, t) ∈ B1 (s , t ) =⇒ #K ((s, t)) = N. (A.31) ∗ ∗ S S ∗ ∗ Moreover, for any (sν, tν) → (s , t ), we have that the sequence K ((sν, tν)) converges to K ((s , t )) as subsets. ∗ ∗ Indeed, ﬁnd any (s , t ) ∈ B0 (s0, t0) which realizes the lower bound N. If not the case, then ∗ ∗ S there exists a sequence (sν, tν) converging to (s , t ) but #K (sν, tν) > N. If the sequence of points S K (sν, tν) have accumulations, then it will contradict with (3); if they don’t accumulate, then by choosing a subsequence, we see that #KS(s∗, t∗) ≥ N + 1 which contradicts with the choice of (s∗, t∗). ∗ ∗ II. c) Hence we can actually assume that (s0, t0) = (s , t ) and 1 = 0. In particular, we take s1, s2, . . . , sN ∈ [−T,T ] be all numbers such that

u(s0, t0) ∈ Gu(si, t0), i = 1,...,N. We claim that (which is similar to that in [11, Page 262]) there exists δ > 0 and r > 0 such that N 0 u (B2δ(s0, t0)) ⊂ G ∪j=1u(Br(sj, t0)) , j 6= j =⇒ Br(sj, t0) ∩ Br(sj0 , t0) = ∅. (A.32) Then we deﬁne Σj := (s, t) ∈ Bδ(s0, t0) | u(s, t) ∈ Gcl(u(Br(sj, t))) . (A.33)

These are closed sets and Bδ(s0, t0) = Σ1 ∪· · ·∪ΣN . Then there exists j0 such that (s0, t0) ∈ IntΣj0 .

Then we take a small ρ > 0 such that Bρ(s0, t0) ⊂ IntΣj0 and Bρ(s0, t0) ∩ Br(s1, t0) = ∅. 44 GUANGBO XU

S II. d) For every (s, t) ∈ Bρ(s0, t0), we see that K ((s, t)) ∩ Br(sj0 , t0) contains a unique element (s0, t). By Lemma A.8 we see that for ρ ≤ r and the two objects u(s + ·, t + ·) and u(s + ·, t + ·) e 0 0 e j0 0 are gauge equivalent over Bρ(0). Then by Proposition A.7, there exists a gauge transformation g :Θ → G such that g∗u = u(s − s + ·, ·). e e j0 0 Let δs = s − s 6= 0. Then we see that u and u(kδs + ·, ·) are gauge equivalent for all k ∈ . j0 0 e e Z This implies that ue(s0, t) ∈ Gx±(t) by the asymptotic behavior of ﬁnite energy solutions. This U contradicts with our choice of (s0, t0), which means that RI (u) is indeed dense in ΘI (u). A.3. The universal moduli space over the space of admissible almost complex structures. For l ≥ k ≥ 2, p > 2, we denote

k,p l n l k,p o M [x±]; Je ,H := ([ue],J) | J ∈ Je , [ue] = [u, Φ, Ψ] ∈ M ([x±]; J, H) . (A.34) Proposition A.10. For any l ≥ k, k ≥ 1, the universal moduli space is a Cl−k-Banach submanifold of Bk,p × Jel.

Proof. We just need to prove that the linearized operator

k,p l k−1,p D : T[u]B × TJ Je → E (A.35) e [ue] is surjective. It is equivalent to consider the augmented one, for any representative ue ∈ Mf(x±; J, H), which is

k,p l k−1,p k−1,p D : TuBe × TJ Je → Ee ⊕ W (Θ, g) . (A.36) e ue Now, because the restriction of D to the first component, which is the augmented linearized operator D , is already Fredholm, we only need to prove that D has dense range. If not the ue k−1,p k−1,p case, then there exists a nonzero vector η = (η, ϑ1, ϑ2) ∈ Ee ⊕ W (Θ, g) which lies in the e ue 2 L -orthogonal complement of the image of D; therefore ηe also lies in the orthogonal complement ∗ ∗ of the image of Du. Hence η ∈ kerD . By elliptic regularity associated to D , we see that η is e e ue ue e continuous. We will show that ηe vanishes on an nonempty open subset of RI (u), which, by the unique continuation property of D∗ , contradicts with the fact that η 6= 0. ue e 0 00 Indeed, on RI (u), we decompose η = η + η with respect to the splitting (A.1) (which also 0 00 l depends on t). However, it is easy to find E ,E ∈ TJ Je , supported near the G-orbit of u(z) for any z ∈ R (u), such that E0(s, t) is zero on gC , E00(s, t) is zero on T Jt , and such that D(0,E0) I U M (resp. D(0,E00)) has positive L2-pairing with η0 (resp. η00) (the concrete way of constructing E0 00 and E are similar to that in [24]). Then this shows that η vanishes on RI (u). ∗ Then, looking at the first component of 0 = D (η, ϑ1, ϑ2), we see on RI (u), ue

JtXϑ1 − Xϑ2 = 0. (A.37)

By the deﬁnition of RI (u), we see that ϑ1|RI (u) = ϑ2|RI (u) = 0. This ﬁnishes our proof. Now the projection

l l M [x±]; Je ,H → Je (A.38) is a Fredholm map of class Cl−k. Then by Sard-Smale theorem, we obtain GAUGED FLOER HOMOLOGY 45

Theorem A.11. For any pair [x±] ∈ CritAH , there exists l0 = l0(k, [x±]) ∈ Z+ such that for any l;reg l l;reg l ≥ l0, there exists a Baire subset JeH ⊂ Je , such that for any J ∈ JeH and any [ue] = [u, Φ, Ψ] ∈ M ([x±]; J, H), the linearized operator J,H k,p k−1,p D : T[u]B → E (A.39) [ue] e [ue] is surjective.

Using Taubes’ trick (see [24, Page 52]), we obtain

reg reg Corollary A.12. There exists a Baire subset JeH ⊂ Je such that for any J ∈ JeH , the moduli k,p space M ([x±]; J, H) is a smooth submanifold of B ([x±]) for any k ≥ 1 and p > 2.

A.4. Transversality for continuation map. It is easier to achieve transversality for the moduli space deﬁning the continuation map, because we are allowed to have objects which depend both on s and t. We brieﬂy sketch the method. Suppose we have a given homotopy Hs,t of compactly α β supported Hamiltonians for (s, t) ∈ Θ such that Hs,t = Ht for s << 0 and Hs,t = Ht for s >> 0. Now we are given two regular families of almost complex structures

α reg β reg J ∈ JeHα (M, U, J) ,J ∈ JeHβ (M, U, J) . (A.40) α β Moreover, suppose we have two possibly diﬀerent constants λ , λ > 0, with a homotopy λs with α β λs = λ for s << 0 and λs = λ for s >> 0. We want to show that, for a “generic” homotopy α β from J to J (denoted by a family of almost complex structures Js,t, with Js,· ∈ Je(M, U, J)), for α β each pair [x ] ∈ CritAHα ,[x ] ∈ CritAHβ , the moduli space of solutions to the following equation  ∂u ∂u  + X (u) + J + X (u) − Y (u) = 0;  Φ s,t Ψ Hs,t ∂s ∂t (A.41) ∂Ψ ∂Φ  − + [Φ, Ψ] + λ2µ(u) = 0  ∂s ∂t s α β satisfying the conditions that ue = (u, Φ, Ψ) is asymptotic to x (resp. x ) as s → −∞ (resp. s → +∞), is transverse. |s| Now take the function V (s, t) = e which diverges exponentially as s → ±∞. For l ∈ Z+ ∪{∞}, consider the space Jel J α,J β (A.42)

l consisting of C -family of admissible almost complex structures Js,t (with respect to the same U and J as before) parametrized by (s, t) ∈ Θ, such that

α β l l |V (s, t)(Js,t − Jt )|C (Θ−×M) < ∞, |V (s, t)(Js,t − Jt )|C (Θ+×M) < ∞. (A.43) Those J’s are homotopies that are asymptotic to J α (resp. J β) exponentially with their derivatives up to order l. Then, for each J ∈ Jel(J α,J β), consider the moduli space

α β k,p α β N [x ], [x ]; J , H , λs ⊂ B [x ], [x ] (A.44)

α β of solutions to (A.41). Since all element [ue] ∈ M [x ], [x ]; J , H , λs will go into U as |s| → ∞, this implies that every such [ue] is “irreducible” in the sense of [2]. Hence we can prove the 46 GUANGBO XU

l α β transversality of the universal moduli space over Je J ,J in the same way as in [2], because now the perturbation could be dependent on both s and t. Then using the implicit function theorem and Taubes’ trick, it is easy to prove the following proposition.

Proposition A.13. There exists a Baire subset J reg J α,J β ⊂ J J α,J β such that for ev- eH ,λs e reg α β α β ery ∈ J J ,J and every pair [x ] ∈ CritA α , [x ] ∈ CritA β , the moduli space J eH ,λs H H α β α β N [x ], [x ]; J , H , λs is a smooth manifold of dimension CZ([x ]) − CZ([x ]).

References

[1] Elie´ Cartan, La topologie des espaces représentatifsdes groupes de Lie, L’Enseignement Mathématique 35 (1936), 177–200. [2] Kai Cieliebak, Ana Rita Pires Gaio, Ignasi Mundet i Riera, and Dietmar Salamon, The symplectic vortex equations and invariants of Hamiltonian group actions, Journal of Symplectic Geometry 1 (2002), no. 3, 543–645. [3] Kai Cieliebak, Ana Rita Pires Gaio, and Dietmar Salamon, J-holomorphic curves, moment maps, and invariants of Hamiltonian group actions, International Mathematical Research Notices 16 (2000), 831–882. [4] Simon Donaldson, Floer homology groups in Yang-Mills theory, Cambridge Tracts in Mathematics, vol. 147, Cambridge University Press, Cambridge, 2002. [5] Simon Donaldson and Peter Kronheimer, The geometry of four-manifolds, Oxford Mathematical Monographs, Clarendon Press · Oxford, 1990. [6] Stamatis Dostoglou and Dietmar Salamon, Self-dual instantons and holomorphic curves, Annals of Mathematics 139 (1994), no. 3, 581–640. [7] Andreas Floer, An instanton-invariant for 3-manifolds, Communications in Mathematical Physics 118 (1988), no. 2, 215–240. [8] , Morse theory for Lagrangian intersections, Journal of Differential Geometry 28 (1988), 513–547. [9] , Symplectic fixed points and holomorphic spheres, Communications of Mathematical Physics 120 (1989), 575–611. [10] Andreas Floer and Helmut Hofer, Coherent orientations for periodic orbit problems in symplectic geometry, Mathematische Zeitschrift 212 (1993), no. 1, 13–38. [11] Andreas Floer, Helmut Hofer, and Dietmar Salamon, Transversality in elliptic Morse theory for the symplectic action, Duke Mathematical Journal 80 (1995), no. 1, 251–292. [12] Urs Frauenfelder, Floer homology of symplectic quotients and the Arnold-Givental conjecture, International Math- ematics Research Notices 2004 (2004), no. 42, 2179–2269. [13] Kenji Fukaya, Yong-Geun Oh, Hiroshi Ohta, and Kaoru Ono, Lagrangian Floer theory on compact toric manifolds. I., Duke Mathematical Journal 151 (2010), no. 1, 23–174. [14] , Lagrangian Floer theory on compact toric manifolds II: bulk deformations, Selecta Mathematica 17 (2011), no. 3, 609–711. [15] Kenji Fukaya and Kauro Ono, Arnold conjecture and Gromov-Witten invariants for general symplectic manifolds, Topology 38 (1999), 933–1048. [16] Ana Rita Pires Gaio and Dietmar Salamon, Gromov-Witten invariants of symplectic quotients and adiabatic limits, Journal of Symplectic Geometry 3 (2005), no. 1, 55–159. [17] Mikhael Gromov, Pseudoholomorphic curves in symplectic manifolds, Inventiones Mathematicae 82 (1985), no. 2, 307–347. [18] Victor Guillemin and Steven Sternberg, Symplectic techniques in physics, Cambridge University Press, Cam- bridge, 1984. [19] Helmut Hofer and Dietmar Salamon, Floer homology and Novikov rings, The Floer Memorial Volume, Birkhäuser, 1995, pp. 483–524. [20] Michael Hutchings and Michael Sullivan, Rounding corners of polygons and the embedded contact homology of T 3, Geometry & Topology 10 (2006), 169–266. GAUGED FLOER HOMOLOGY 47

[21] Michael Hutchings and Clifford Taubes, Gluing pseudoholomorphic curves along branched covered cylinders. i., Journal of Symplectic Geometry 5 (2007), no. 1, 43–137. [22] Peter Kronheimer and Tomasz Mrowka, Monopoles and three-manifolds, New Mathematical Monographs, 10, Cambridge University Press, Cambridge, 2007. [23] Gang Liu and Gang Tian, Floer homology and Arnold conjecture, Journal of Differential Geometry 49 (1998), 1–74. [24] Dusa McDuff and Dietmar Salamon, J-holomorphic curves and symplectic topology, Colloquium Publications, vol. 52, American Mathematical Society, Providence, RI, 2004. [25] Ignasi Mundet i Riera, Hamiltonian Gromov-Witten invariants, Topology 43 (2003), no. 3, 525–553. [26] Ignasi Mundet i Riera and Gang Tian, A compactification of the moduli space of twisted holomorphic maps, Advances in Mathematics 222 (2009), 1117–1196. [27] Louis Nirenberg and Shmuel Agmon, Lower bounds and uniqueness theorems for solutions of differential equations in a Hilbert space, Communications on Pure and Applied Mathematics 20 (1967), 207–229. [28] Kauro Ono, The Arnold conjecture for weakly monotone symplectic manifolds, Inventionne Mathematicae 119 (1995), 519–537. [29] Peter Ozsváthand ZoltánSzabó, Holomorphic disks and topological invariants for closed three-manifolds, Annals of Mathematics 159 (2004), no. 3, 1027–1158. [30] Sergey Piunikhin, Dietmar Salamon, and Matthias Schwarz, Symplectic Floer-Donaldson theory and quantum cohomology, Contact and Symplectic Geometry, Publications of the Newton Institute, Cambridge University Press, Cambridge, 1996. [31] Joel Robbin and Dietmar Salamon, The Maslov index for paths, Topology 32 (1993), no. 4, 827–844. [32] Dietmar Salamon, Lectures on Floer homology, Symplectic geometry and topology, American Mathematical Society, Providence, RI, 1997. [33] Stephen Schecter and Guangbo Xu, Morse theory for Lagrange multipliers and adiabatic limits, http://arxiv. org/abs/1211.3028, 2012. [34] Edward Witten, Supersymmetry and Morse theory, Journal of Differential Geometry 17 (1982), no. 4, 661–692. [35] Chris Woodward, Quantum Kirwan morphism and Gromov-Witten invariants of quotients, arXiv: 1204.1765. [36] , Gauged Floer theory for toric moment fibers, Geometric and Functional Analysis 21 (2011), 680–749, arxiv.1004.2841. [37] Guangbo Xu, Moduli space of twisted holomorphic maps with Lagrangian boundary condition: compactness, Advances in Mathematics 242 (2013), 1–49. [38] Fabian Ziltener, A quantum Kirwan map: bubbling and Fredholm theory, To appear in Memoirs of the American Mathematical Society, 2012.

Department of Mathematics, 410N Rowland Hall, University of California, Irvine, Irvine, CA 92697 USA E-mail address: [email protected]