Pacific Journal of Mathematics Vol 230 Issue 2, Apr 2007

PACIFIC JOURNAL OF MATHEMATICS Mathematics of Journal Paciﬁc

Volume 230 No. 2 April 2007 Paciﬁc Journal of Mathematics 2007 o.20 o 2 No. 230, Vol.

Volume 230 No. 2 April 2007 PACIFIC JOURNAL OF MATHEMATICS http://www.pjmath.org

Founded in 1951 by E. F. Beckenbach (1906–1982) F. Wolf (1904–1989)

EDITORS V. S. Varadarajan (Managing Editor) Department of Mathematics University of California Los Angeles, CA 90095-1555 paciﬁ[email protected]

Vyjayanthi Chari Darren Long Sorin Popa Department of Mathematics Department of Mathematics Department of Mathematics University of California University of California University of California Riverside, CA 92521-0135 Santa Barbara, CA 93106-3080 Los Angeles, CA 90095-1555 [email protected] [email protected] [email protected] Robert Finn Jiang-Hua Lu Jie Qing Department of Mathematics Department of Mathematics Department of Mathematics Stanford University The University of Hong Kong University of California Stanford, CA 94305-2125 Pokfulam Rd., Hong Kong Santa Cruz, CA 95064 ﬁ[email protected] [email protected] [email protected] Kefeng Liu Alexander Merkurjev Jonathan Rogawski Department of Mathematics Department of Mathematics Department of Mathematics University of California University of California University of California Los Angeles, CA 90095-1555 Los Angeles, CA 90095-1555 Los Angeles, CA 90095-1555 [email protected] [email protected] [email protected]

PRODUCTION paciﬁ[email protected] Paulo Ney de Souza, Production Manager Silvio Levy, Senior Production Editor Alexandru Scorpan, Production Editor

SUPPORTING INSTITUTIONS

ACADEMIASINICA, TAIPEI UNIVERSIDADDELOSANDES UNIV. OFCALIF., SANTA CRUZ CALIFORNIAINST. OFTECHNOLOGY UNIV. OFARIZONA UNIV. OF HAWAII INST. DE MATEMÁTICA PURA E APLICADA UNIV. OFBRITISHCOLUMBIA UNIV. OF MONTANA KEIOUNIVERSITY UNIV. OFCALIFORNIA, BERKELEY UNIV. OF NEVADA, RENO MATH. SCIENCESRESEARCHINSTITUTE UNIV. OFCALIFORNIA, DAVIS UNIV. OFOREGON NEW MEXICO STATE UNIV. UNIV. OFCALIFORNIA, IRVINE UNIV. OFSOUTHERNCALIFORNIA OREGON STATE UNIV. UNIV. OFCALIFORNIA, LOSANGELES UNIV. OF UTAH PEKINGUNIVERSITY UNIV. OFCALIFORNIA, RIVERSIDE UNIV. OF WASHINGTON STANFORD UNIVERSITY UNIV. OFCALIFORNIA, SANDIEGO WASHINGTON STATE UNIVERSITY UNIV. OFCALIF., SANTA BARBARA These supporting institutions contribute to the cost of publication of this Journal, but they are not owners or publishers and have no respon- sibility for its contents or policies. See inside back cover or www.pjmath.org for submission instructions. Regular subscription rate for 2007: $425.00 a year (10 issues). Special rate: $212.50 a year to individual members of supporting institutions. Subscriptions, requests for back issues from the last three years and changes of subscribers address should be sent to Pacific Journal of Mathematics, P.O. Box 4163, Berkeley, CA 94704-0163, U.S.A. Prior back issues are obtainable from Periodicals Service Company, 11 Main Street, Germantown, NY 12526-5635. The Pacific Journal of Mathematics is indexed by Mathematical Reviews, Zentralblatt MATH, PASCAL CNRS Index, Referativnyi Zhurnal, Current Mathematical Publications and the Science Citation Index. The Pacific Journal of Mathematics (ISSN 0030-8730) at the University of California, c/o Department of Mathematics, 969 Evans Hall, Berkeley, CA 94720-3840 is published monthly except July and August. Periodical rate postage paid at Berkeley, CA 94704, and additional mailing offices. POSTMASTER: send address changes to Pacific Journal of Mathematics, P.O. Box 4163, Berkeley, CA 94704-0163. PUBLISHED BY PACIFIC JOURNAL OF MATHEMATICS at the University of California, Berkeley 94720-3840 A NON-PROFIT CORPORATION Typeset in LATEX Copyright ©2007 by Pacific Journal of Mathematics PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

A RESULT ABOUT C3-RECTIFIABILITY OF LIPSCHITZ CURVES

SILVANO DELLADIO

1+k Let γ0 :[a, b] → ޒ be Lipschitz. Our main result provides a sufﬁcient condition, expressed in terms of further accessory Lipschitz maps, for the 3 C -rectiﬁability of γ0([a, b]).

1. Introduction

A set in ޒn is C3-rectiﬁable if Ᏼ1-almost all of it can be covered by countably many curves of class C3 embedded in ޒn. The main goal of this paper is to prove the following result. Theorem 1.1. Let there be given Lipschitz maps

1+k 1+k 1+k γ0, γ1 :[a, b] → ޒ and γ2 = (γ2>, γ2⊥) :[a, b] → ޒ × ޒ and a function ω :[a, b] → {±1} such that 0 0 (1-1) γ0(t) = ω(t) kγ0(t)k γ1(t), 0 0 0 0 (1-2) (γ0(t), γ1(t)) = ω(t) k(γ0(t), γ1(t))k γ2(t) 3 for almost every t ∈ [a, b]. Then γ0([a, b]) is a C -rectiﬁable set.

Remark. In the special case when ω := 1 while γ0 is regular and at least of class 2 C , the conditions (1-1) and (1-2) say that γ1(t) and γ2(t) are, respectively, the unit tangent vector of γ0 at t and the unit tangent vector of (γ0, γ1) at t. This remark is at the root of the applications to geometric variational problems mentioned below. Theorem 1.1 should be considered a step forward in a project, stated in [Delladio 2005], aimed at providing sufficient conditions for the C H -rectifiability of a n- dimensional rectifiable set. Results concerning the case H = 2 were first obtained in [Anzellotti and Serapioni 1994; Delladio 2003; Fu 1998], but subtle mistakes seriously invalidating their proofs were discovered later [Delladio 2004; Fu 2004]. Then the paper [Delladio 2005], cleaning-up the simplest case n = 1 and H = 2, followed. Our future efforts will be aimed at extending the theory to any value of n

MSC2000: primary 49Q15, 53A04; secondary 26A12, 26A16, 28A75, 28A78, 54C20. Keywords: rectiﬁable sets, geometric measure theory, Whitney extension theorem.

257 258 SILVANO DELLADIO and H. Joint work with Joseph Fu on C2-rectifiability in all dimensions (invoking slicing in order to reduce to dimension one) is in progress. Further work is in progress to apply these results to geometric variational problems via geometric measure theory and more precisely through the notion, first introduced in [Anzellotti et al. 1990], of a generalized Gauss graph. Former achieve- ments in this direction include [Delladio 2001] (a somehow surprising application to differential geometry context), [Anzellotti and Delladio 1995] (an application to Willmore problem) and [Delladio 1997] (an application to a problem introduced in [Bellettini et al. 1993]). These last two papers followed the idea by De Giorgi of relaxing the functional with respect to L1-convergence of the domains of integration. Now we expect that our results can be applied to handle functionals with integrands involving curvatures with their derivatives and, in particular, to get explicit representation formulas after relaxation. 2 The proof of Theorem 1.1 starts from the C -rectifiability of γ0([a, b]), which is guaranteed by condition (1-1), as shown in [Delladio 2005]. The problem is 2 reduced in Section 2 to proving that γ0([a, b]) intersects the graph of any C map

f : ޒ → (ޒu)⊥ (u ∈ ޒ1+k, kuk = 1) in a C3-rectifiable set. From the first and second derivatives of f expressed in terms of the γi , we obtain in Section 3 a second order Taylor-type formula for f with the remainder in terms of the γi . Theorem 1.1 then follows by the Whitney Extension Theorem, also involving a Lusin-type argument (Section 4). Finally, the absolute curvature for a one-dimensional C2-rectifiable set P is defined and proved to be approximately differentiable almost everywhere whenever P is C3-rectifiable (Section 5).

2. Reduction to graphs

By virtue of the main result stated in [Delladio 2005], the equality (1-1) implies 2 that γ0([a, b]) is C -rectiﬁable. As a consequence, there must be countably many unit vectors 1+k u j ∈ ޒ and maps of class C2 ⊥ f j : ޒ → (ޒu j ) such that 1 [ ] \ S = Ᏼ γ0( a, b ) j G f j 0 where := + | ∈ G f j xu j f j (x) x ޒ . A RESULT ABOUT C3-RECTIFIABILITYOFLIPSCHITZCURVES 259

[ ] ∩ 3 Hence we need only show that the sets γ0( a, b ) G f j are C -rectiﬁable. In other words, Theorem 1.1 becomes an immediate corollary of the following result.

Theorem 2.1. Let γ0, γ1, γ2 be as in Theorem 1.1. Consider a map f : ޒ → (ޒu)⊥ (u ∈ ޒ1+k, kuk = 1) of class C2 and define G f := {xu + f (x) | x ∈ ޒ}. 3 Then the set G f ∩ γ0([a, b]) is C -rectifiable. In this section we take the first step toward the proof of Theorem 2.1, which will be concluded later in Section 4. Define := −1 ∩ ∈ [ ] 0 0 0 6= L γ0 (G f ) t a, b γ0(t), γ1(t) exist, γ0(t) 0, (1-1) and (1-2) hold . By Lusin’s Theorem, for any given real number ε > 0, there exists a closed subset Lε of L such that 0 1 (2-1) γ0|Lε and ω|Lε are continuous and ᏸ (L\Lε) ≤ ε. ∗ If Lε denotes the set of the density points of Lε, then ∗ (2-2) Lε ⊂ Lε since Lε is closed. The equality 1 ∗ (2-3) ᏸ (Lε\Lε ) = 0 also holds by a celebrated result of Lebesgue. In the special case that L has measure ∗ zero, we take Lε := ∅, hence Lε := ∅. Now observe that ∩ [ ] \ ∗ ⊂ −1 ∩ [ ]\ ∗ G f γ0( a, b ) γ0(Lε ) γ0 γ0 (G f ) a, b Lε hence 1 ∩ [ ] \ ∗ ≤ 1 −1 ∩ [ ]\ ∗ Ᏼ G f γ0( a, b ) γ0(Lε ) Ᏼ γ0 γ0 (G f ) a, b Lε Z Z 0 0 ≤ kγ0k = kγ0k ≤ ε Lip(γ0), −1 ∩[ ]\ ∗ \ ∗ γ0 (G f ) a,b Lε L Lε which implies 1 ∩ [ ] \ S∞ ∗ = Ᏼ G f γ0( a, b ) j=1 γ0(L1/j ) 0. Hence, to prove Theorem 2.1, it will be enough to verify that

∗ 3 (2-4) γ0(Lε ) is C -rectiﬁable for all ε > 0. 260 SILVANO DELLADIO

3. Second order Taylor formula and estimates

Proposition 3.1 below gives formulas for the ﬁrst and second derivatives of f in terms of the γi . This yields a suitable second order Taylor formula in Theorem 3.1. Throughout this section we shall assume ᏸ1(L) > 0. Notice that

(3-1) γ2>(s) 6= 0 for all s ∈ L by (1-2), so the map

1+k γ2⊥(t) µ : t ∈ [a, b] γ2>(t) 6= 0 → ޒ , µ(t) := kγ2>(t)k is well-deﬁned in L. Lemma 3.1. Let A, B, u ∈ ޒ1+k, with kuk = 1. Then

(A ∧ B) u = (A · u)B − (B · u)A.

1+k Proof. Let {e j } be an orthonormal basis of ޒ such that e1 = u. One has X (A ∧ B) u · ei = hA ∧ B, u ∧ ei i = (A j Bl − Al B j )he j ∧ el , e1 ∧ ei i j

x(t) := γ0(t) · u, t ∈ ޒ. ∗ Then, for all s ∈ Lε , one has 0 0 (3-2) x (s) = γ0(s) · u 6= 0 (that is, γ1(s) · u 6= 0) and γ (s) (3-3) f 0(x(s)) = 1 − u. γ1(s) · u Moreover γ1(s) ∧ µ(s) u f 00(x(s)) = . (3-4) 3 (γ1(s) · u) Proof. Observe that

f (x(t)) = γ0(t) − (γ0(t) · u)u = γ0(t) − x(t)u ∈ −1 ∗ for all t γ0 (G f ). The sides of this equality are both differentiable in Lε and ∗ ⊂ −1 ⊂ −1 since each point in Lε γ0 (G f ) is a limit point of Lε γ0 (G f ), the derivatives A RESULT ABOUT C3-RECTIFIABILITYOFLIPSCHITZCURVES 261

∗ have to coincide in Lε . Thus 0 0 0 0 0 0 (3-5) x (s) f (x(s)) = γ0(s) − (γ0(s) · u)u = γ0(s) − x (s)u ∈ ∗ 0 6= ∈ ∗ for all s Lε . We obtain (3-2) by recalling that γ0(s) 0 at all s Lε . Formula (3-3) follows at once from (3-5) and (1-1). ∗ By virtue of (3-2), the sides of (3-3) are both differentiable in Lε . The derivatives ∗ ∗ ∗ must coincide in Lε , since each point of Lε is a limit point of Lε . In view of Lemma 3.1, we then get (γ (s) · u)γ 0(s) − (γ 0(s) · u)γ (s) (γ (s) ∧ γ 0(s)) u x0(s) f 00(x(s)) = 1 1 1 1 = 1 1 2 2 (γ1(s) · u) (γ1(s) · u) ∗ for all s ∈ Lε . Formula (3-4) finally follows from (3-2), (1-1) and (1-2). Now set 1s(t) := γ0(t) − γ0(s), s, t ∈ [a, b]. The map (1 (t) · u)2 6 (t) := 1 (t) − 1 (t) · γ (s) γ (s) − s µ(s), t ∈ [a, b], s s ( s 1 ) 1 2 2(γ1(s) · u) ∗ is well-defined for any given s ∈ Lε , by Proposition 3.1. ∗ If s ∈ Lε , hence s ∈ (a, b) and (3-1) holds, one has k k ≥ 1 k k ∈ γ2>(σ ) 2 γ2>(s) > 0 for all σ Is, where Is denotes a certain nontrivial open interval centered at s and included in [a, b], existing by the continuity of γ2>. In particular, this inequality shows that µ|Is is Lipschitz, so the map given, for σ ∈ Is, by µ(s) 9 (σ):=µ(σ)−(µ(σ)·γ (s))γ (s)− (γ (σ)·u)2+(1 (σ)·u)(µ(σ)·u) s 1 1 2 1 s (γ1(s)·u) ∗ is well-defined and Lipschitz, provided s ∈ Lε . Moreover

9s(s) = 0, as follows at once from (1-2) and from the following simple result. ∈ ∗ · 0 = Proposition 3.2. If s Lε then γ1(s) γ1(s) 0.

Proof. Let {s j } be a sequence in Lε converging to s, with s j 6= s for all j. Since

kγ1(s j )k = kγ1(s)k = 1 for all j, by (1-1) and (2-2), we have 2 2 kγ1(s j )k − kγ1(s)k γ1(s j ) − γ1(s) 0 = = · γ1(s j ) + γ1(s) . s j − s s j − s 262 SILVANO DELLADIO

The conclusion follows by letting j → ∞. ∗ Theorem 3.1. Let s ∈ Lε . ∈ −1 (1) For all t γ0 (G f ), − − 0 − − 1 00 − 2 (3-6) f (x(t)) f (x(s)) f (x(s)) x(t) x(s) 2 f (x(s)) x(t) x(s) 1 = (γ1(s) ∧ 6s(t)) u. γ1(s) · u (2) For all t ∈ Is, Z t Z ρ 0 0 6s(t) = ω(ρ)kγ0(ρ)k ω(σ )kγ0(σ )k9s(σ ) dσ dρ. s s Proof. (1) By recalling Proposition 3.1 and Lemma 3.1, we get

− − 0 − − 1 00 − 2 f (x(t)) f (x(s)) f (x(s))(x(t) x(s)) 2 f (x(s))(x(t) x(s)) γ1(s) = γ0(t) − x(t)u − (γ0(s) − x(s)u) − − u (x(t) − x(s)) γ1(s) · u

(γ (s) ∧ γ ⊥(s)) u − 1 2 (x(t) − x(s))2 3 2kγ2>(s)k(γ1(s) · u) γ (s) (γ (s) ∧ γ ⊥(s)) u = 1 (t) − 1 (1 (t) · u) − 1 2 (1 (t) · u)2 s s 3 s γ1(s) · u 2kγ2>(s)k(γ1(s) · u)

1 (γ (s) ∧ γ ⊥(s)) u = (γ (s) ∧ 1 (t)) u − 1 2 (1 (t) · u)2 . 1 s 2 s γ1(s) · u 2kγ2>(s)k(γ1(s) · u)

This is just (3-6), in view of the deﬁnition of 6s(t).

(2) Since 1s is Lipschitz and 1s(s) = 0, one has Z t µ(s) d 6 (t) = γ 0(ρ) − (γ 0(ρ) · γ (s))γ (s) − (1 (ρ) · u)2dρ s 0 0 1 1 2 s s 2(γ1(s) · u) dρ Z t µ(s) = γ 0(ρ) − (γ 0(ρ) · γ (s))γ (s) − (1 (ρ) · u)(γ 0(ρ) · u) dρ 0 0 1 1 2 s 0 s (γ1(s) · u) namely Z t 0 (3-7) 6s(t) = ω(ρ)kγ0(ρ)k8s(ρ) dρ s by (1-1), where 8s is the Lipschitz map deﬁned by µ(s) 8 (ρ) := γ (ρ) − (γ (ρ) · γ (s))γ (s) − (1 (ρ) · u)(γ (ρ) · u) s 1 1 1 1 2 s 1 (γ1(s) · u) A RESULT ABOUT C3-RECTIFIABILITYOFLIPSCHITZCURVES 263 for ρ ∈ [a, b]. Observe that

0 0 0 0 kγ0(σ )kγ2⊥(σ ) = k(γ0(σ ), γ1(σ ))kkγ2>(σ )kγ2⊥(σ ) = ω(σ )kγ2>(σ )kγ1(σ ) for a.e. σ ∈ [a, b], by (1-2). Hence

0 0 γ2⊥(σ ) 0 γ1(σ ) = ω(σ )kγ0(σ )k = ω(σ )kγ0(σ )kµ(σ ) kγ2>(σ )k for a.e. σ ∈ [a, b] such that γ2>(σ ) 6= 0 — in particular, for a.e. σ ∈ Is. By recalling the deﬁnition of 9s, it follows at once that

0 0 (3-8) 8s(σ ) = ω(σ )kγ0(σ )k9s(σ ) for a.e. σ ∈ Is. We conclude using (3-7), (3-8) and noting that 8s(s) = 0. 0 As a consequence, we get the following integral representation of 6s and the related ﬁrst order Taylor formula for f 0.

∗ ∗ Corollary 3.1. Let s ∈ Lε and t ∈ Lε ∩ Is. Then

(1) The map 6s is differentiable at t and Z t 0 0 0 6s(t) = ω(t)kγ0(t)k ω(σ )kγ0(σ )k9s(σ ) dσ. s (2) One has f 0(x(t)) − f 0(x(s)) − f 00(x(s))(x(t) − x(s)) Z t 1 0 = γ1(s) ∧ ω(σ )kγ0(σ )k9s(σ )dσ u. (γ1(t) · u) (γ1(s) · u) s

Proof. (1) Observe that t + h ∈ Is ⊂ (a, b) provided |h| is small enough. By Theorem 3.1(2), then, Z t+h Z ρ 6s(t + h) − 6s(t) 1 0 0 = ω(ρ)kγ0(ρ)k ω(σ )kγ0(σ )k9s(σ ) dσ dρ h h t s

= I1(h) + I2(h) for all small enough values of |h|, where we have set — with a harmless abuse of ∗ notation and recalling that ω|Lε is continuous, by (2-1) and (2-2) — Z Z ρ ω(t) 0 0 I1(h) := kγ0(ρ)k ω(σ )kγ0(σ )k9s(σ )dσ dρ, h ∗ [t,t+h]∩Lε s Z Z ρ 1 0 0 I2(h) := ω(ρ)kγ0(ρ)k ω(σ )kγ0(σ )k9s(σ )dσ dρ. h ∗ [t,t+h]\Lε s 264 SILVANO DELLADIO

We have Z Z ρ ω(t) 0 0 0 I1(h) = kγ0(ρ)k − kγ0(t)k ω(σ )kγ0(σ )k9s(σ ) dσ dρ h ∗ [t,t+h]∩Lε s ω(t)kγ 0(t)k + 0 h Z Z t Z ρ 0 0 × ω(σ )kγ0(σ )k9s(σ )dσ + ω(σ )kγ0(σ )k9s(σ ) dσ dρ. ∗ [t,t+h]∩Lε s t Recalling that 0| ∗ (i) γ0 Lε is continuous, by (2-1) and (2-2),

(ii) γ0 is Lipschitz and 9s is bounded (in fact it is Lipschitz!), and ∗ (iii) t is a density point of Lε (hence of Lε , by (2-3)), we see that Z t 0 0 lim I1(h) = ω(t)kγ0(t)k ω(σ )kγ0(σ )k9s(σ ) dσ. h→0 s The conclusion follows now by observing that, as an easy consequence of (ii) and (iii), one also has lim I2(h) = 0. h→0 (2) The two members of (3-6) are differentiable at t, by (1). Since t is a limit ⊂ −1 point of Lε γ0 (G f ) the derivatives have to coincide, by Theorem 3.1(1), namely

0 0 00 0 1 0 f (x(t))− f (x(s))− f (x(s)) x(t)−x(s) x (t) = γ1(s)∧6s(t) u. γ1(s)·u We conclude by recalling Proposition 3.1, part (1) of the corollary and (1-1).

4. Conclusion of the proof of Theorem 1.1

To complete the proof of Theorem 2.1, hence of Theorem 1.1, we have to verify (i) ∗ (2-4). For i = 1, 2,... , deﬁne 0 as the set of the points s ∈ Lε satisfying, for all ∗ t ∈ Lε such that |t − s| ≤ (b − a)/i, the estimates − − 0 − − 1 00 − 2 f (x(t)) f (x(s)) f (x(s))(x(t) x(s)) 2 f (x(s))(x(t) x(s)) ≤ i |x(t) − x(s)|3, k f 0(x(t)) − f 0(x(s)) − f 00(x(s))(x(t) − x(s))k ≤ i |x(t) − x(s)|2, k f 00(x(t)) − f 00(x(s))k ≤ i |x(t) − x(s)|. Obviously, (i) (i+1) ∗ 0 ⊂ 0 ⊂ Lε A RESULT ABOUT C3-RECTIFIABILITYOFLIPSCHITZCURVES 265 for all i, and it is easy to verify that

S (i) ∗ (4-1) i 0 = Lε ; ∗ indeed, for s ∈ Lε , Theorem 3.1 and the equality 9s(s) = 0 for the Lipschitz function 9s imply that − − 0 − − 1 00 − 2 f (x(t)) f (x(s)) f (x(s))(x(t) x(s)) 2 f (x(s))(x(t) x(s)) 2 Z t Z ρ k6s(t)k Lip(γ0) Lip(9s) ≤ ≤ |σ − s| dσ dρ |γ1(s) · u| |γ1(s) · u| s s = A(s) |t − s|3

∗ for all t ∈ Lε ∩ Is, where Lip(γ )2 Lip(9 ) A(s) := 0 s . 6 |γ1(s) · u| Since x(t) − x(s) → x0(s)(as t → s) t − s and x0(s) 6= 0 by Proposition 3.1, it follows that 0 x(t) − x(s) |x (s)| (4-2) ≥ > 0 t − s 2 provided |t − s| is small enough. Then

− − 0 − − 1 00 − 2 (4-3) f (x(t)) f (x(s)) f (x(s))(x(t) x(s)) 2 f (x(s))(x(t) x(s)) 8A(s) ≤ |x(t) − x(s)|3 |x0(s)|3 ∗ whenever t lies in Lε and |t − s| is small enough. Analogously, from Corollary 3.1(2) we get k f 0(x(t)) − f 0(x(s)) − f 00(x(s))(x(t) − x(s))k Z t 1 0 ≤ ω(σ )kγ0(σ )k9s(σ )dσ |γ1(t) · u| |γ1(s) · u| s Z t Lip(γ0) Lip(9s) B(s) 2 ≤ |σ − s| dσ = |t − s| |γ1(t) · u| |γ1(s) · u| s |γ1(t) · u| ∗ for all t ∈ Lε ∩ Is, where Lip(γ ) Lip(9 ) B(s) := 0 s . 2 |γ1(s) · u| 266 SILVANO DELLADIO

Since γ1(t) → γ1(s) as t → s and since γ1(s) · u 6= 0 by Proposition 3.1, one also has |γ (s) · u| (4-4) |γ (t) · u| ≥ 1 > 0 1 2 provided |t − s| is small enough. Recalling (4-2), we obtain

(4-5) k f 0(x(t)) − f 0(x(s)) − f 00(x(s))(x(t) − x(s))k 8 B(s) ≤ |x(t) − x(s)|2 0 2 |γ1(s) · u| |x (s)| ∗ on condition that t ∈ Lε and |t − s| is small enough. Since µ|Is is Lipschitz and by (4-4), it follows that the map γ1(t) ∧ µ(t) u t 7→ 3 (γ1(t) · u) is Lipschitz in a neighborhood of s. Then, by also recalling Proposition 3.1, a number C(s) has to exist such that

k f 00(x(t)) − f 00(x(s))k ≤ C(s) |t − s|

(i) 3 (4-7) γ0(0 ) is C -rectiﬁable for all i.

To prove this, we ﬁrst set (b − a) j a(i) := a + ( j = 0,..., i), j i (i) := (i) ∩ (i) (i) = − 0 j 0 a j , a j+1 ( j 0,..., i 1), (i) (i) Fj := x 0 j ( j = 0,..., i − 1).

(i) (i) For any pair ξ, η ∈ Fj , there are two sequences {sh}, {th} ⊂ 0 j such that

lim x(sh) = ξ, lim x(th) = η. h→∞ h→∞ A RESULT ABOUT C3-RECTIFIABILITYOFLIPSCHITZCURVES 267

Since the three estimates at the beginning of this section (page 264) hold with s = sh, t = th, we obtain, by letting h → ∞, − − 0 − − 1 00 − 2 ≤ | − |3 f (η) f (ξ) f (ξ)(η ξ) 2 f (ξ)(η ξ) i η ξ , 0 0 00 2 f (η) − f (ξ) − f (ξ)(η − ξ) ≤ i |η − ξ| , 00 00 f (η) − f (ξ) ≤ i |η − ξ|.

(i) 2,1 Therefore f |Fj can be extended to a map of class C

(i) ⊥ f j : ޒ → (ޒu) by invoking the Whitney extension Theorem [Stein 1970, Chapter VI, §2.3]. (i) Finally, a Lusin-type result [Federer 1969, §3.1.15] implies that γ0(0 j ) has to be C3-rectiﬁable (compare [Anzellotti and Serapioni 1994, Proposition 3.2]). Hence (4-7) follows.

5. Approximately differentiable absolute curvature of a one-dimensional C3-rectiﬁable set

We now extend the notion of absolute curvature to arbitrary one-dimensional C2- rectiﬁable subsets P of ޒ1+k. Consider a “C2-covering of P”, that is, a countable family

Ꮽ = {Ci }, 2 where the Ci are compact curves of class C , embedded in the base space and such that 1 S Ᏼ P \ i Ci = 0. Part (1) of the next proposition and the remark following it provide the argument proving the well-posedness of Deﬁnition 5.1 below.

1+k 2 Proposition 5.1. Let ϕ, ψ : ޒ → ޒ be maps of class C and let x0 be a density point of F := x ∈ ޒ ϕ(x) = ψ(x) . 0 0 00 00 (1) ϕ (x0) = ψ (x0) and ϕ (x0) = ψ (x0). 000 000 3 (2) ϕ (x0) = ψ (x0) if ϕ and ψ are of class C .

Proof. The set F∗ of density points of F satisﬁes F∗ ⊂ F and ᏸ1(F\F∗) = 0; ∗ ∗ hence every point in F is a limit point of F . The proposition follows. Remark. The following facts follow easily from Proposition 5.1(1). 268 SILVANO DELLADIO

(a) If x is a density point of both P ∩Ci and P ∩C j , then the absolute curvatures ∗ of Ci and C j coincide at x. Hence, denoting by (P ∩ Ci ) the set of density points of P ∩ Ci , the function

Ꮽ S ∗ αP : i (P ∩ Ci ) → ޒ, x 7→ the absolute curvature of Ci(x) at x ∗ where i(x) is any index such that x ∈ (P ∩Ci(x)) , is well-deﬁned. Moreover,

1 S ∗ 1 S 1 S Ᏼ P \ i (P ∩ Ci ) = Ᏼ P \ i (P ∩ Ci ) = Ᏼ P \ i Ci = 0, by a well-known result of Lebesgue. 2 Ꮽ Ꮾ (b) If Ꮾ is another C -covering of P, then αP and αP are representatives of the same measurable function, with domain P. Deﬁnition 5.1. The measurable real-valued function with domain P and having Ꮽ αP as a representative (see preceding remark) is said to be the absolute curvature of P and is denoted by αP . 3 Proposition 5.2. If P is C -rectiﬁable, then αP is approximately differentiable; that is:

3 Ꮽ (1) For any given C -covering Ꮽ = {Ci } of P, the function αP is approximately ∗ differentiable at every point in (P ∩ Ci ) , for all i. (2) If Ꮽ and Ꮾ are C3-coverings of P, then one has

Ꮽ Ꮾ apDαP = apDαP , a.e. in P. ∈ ∩ ∗ Proof. (1) Consider any point a (P Ci0 ) . Without loss of generality, we can 3 assume that Ci0 is the graph of a function of class C , namely = { + | ∈ } Ci0 tu h(t) t I where u is a unit vector in ޒ1+k, I is a closed interval centered at 0 and

h ∈ C3(I ,(ޒu)⊥), h(0) = a.

Set U := I ◦ × ޒk and let g : U → ޒ be deﬁned as the function mapping (t, v) ∈ U + to the absolute curvature of Ci0 at tu h(t), that is, k(u + h0(t)) ∧ h00(t)k (5-1) g(t, v) = ,(t, v) ∈ U (1 + kh0(t)k2)3/2 by (8.4.13.1) in [Berger and Gostiaux 1988]. Obviously, the function g is differentiable at a. Moreover, since ∩ ∗ ⊂ := ∈ S ∩ ∗ Ꮽ = (P Ci0 ) E x i (P Ci ) αP (x) g(x) A RESULT ABOUT C3-RECTIFIABILITYOFLIPSCHITZCURVES 269

Ꮽ by the deﬁnition of αP , the set E has density 1 at a. According to [Federer 1969, Ꮽ §3.2.16], the function αP is approximately differentiable at a and one has

Ꮽ 0 (5-2) apDαP (a) = Dg(a)|ޒτ, with τ := (1, h (0)).

(2) This follows easily from (5-1) and (5-2), by recalling Proposition 5.1.

References

[Anzellotti and Delladio 1995] G. Anzellotti and S. Delladio, “Minimization of functionals of curvatures and the Willmore problem”, pp. 33–43 in Advances in geometric analysis and continuum mechanics (Stanford, 1993), edited by P. Concus and K. Lancaster, International Press, Cambridge, MA, 1995. MR 96i:49052 Zbl 0840.49008 [Anzellotti and Serapioni 1994] G. Anzellotti and R. Serapioni, “Ꮿk-rectifiable sets”, J. Reine Angew. Math. 453 (1994), 1–20. MR 95g:49078 Zbl 0799.49028 [Anzellotti et al. 1990] G. Anzellotti, R. Serapioni, and I. Tamanini, “Curvatures, functionals, currents”, Indiana Univ. Math. J. 39:3 (1990), 617–669. MR 91k:49059 Zbl 0718.49030 [Bellettini et al. 1993] G. Bellettini, G. Dal Maso, and M. Paolini, “Semicontinuity and relaxation properties of a curvature depending functional in 2D”, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 20:2 (1993), 247–297. MR 94g:49101 Zbl 0797.49013 [Berger and Gostiaux 1988] M. Berger and B. Gostiaux, Differential geometry: manifolds, curves, and surfaces, Graduate Texts in Mathematics 115, Springer, New York, 1988. MR 88h:53001 Zbl 0629.53001 [Delladio 1997] S. Delladio, “Special generalized Gauss graphs and their application to minimization of functionals involving curvatures”, J. Reine Angew. Math. 486 (1997), 17–43. MR 98f:49043 Zbl 0871.49034 [Delladio 2001] S. Delladio, “On hypersurfaces in Rn+1 with integral bounds on curvature”, J. Geom. Anal. 11:1 (2001), 17–42. MR 2002a:49059 Zbl 1034.49043 [Delladio 2003] S. Delladio, “The projection of a rectifiable Legendrian set is C2-rectifiable: a simplified proof”, Proc. Roy. Soc. Edinburgh Sect. A 133:1 (2003), 85–96. MR 2003m:53089 Zbl 1035.53010 [Delladio 2004] S. Delladio, “Taylor polynomials and non-homogeneous blow-ups”, Manuscripta Math. 113:3 (2004), 383–395. MR 2006a:41007 Zbl 02078018 [Delladio 2005] S. Delladio, “A result about C2-rectifiability of one-dimensional rectifiable sets: Application to a class of one-dimensional integral currents”, preprint, Università di Trento, 2005, Available at http://eprints.biblio.unitn.it/archive/00000783. To appear in Boll. Un. Matem. Italiana. [Federer 1969] H. Federer, Geometric measure theory, Grundlehren der math. Wiss. 153, Springer, New York, 1969. MR 41 #1976 Zbl 0176.00801 [Fu 1998] J. H. G. Fu, “Some remarks on Legendrian rectifiable currents”, Manuscripta Math. 97:2 (1998), 175–187. MR 2000g:49051 Zbl 0916.53038 [Fu 2004] J. H. G. Fu, “Erratum to: “Some remarks on Legendrian rectifiable currents” [Manuscripta Math. 97 (1998), no. 2, 175–187]”, Manuscripta Math. 113:3 (2004), 397–401. MR 2005k:49120 Zbl 1066.53014 [Stein 1970] E. M. Stein, Singular integrals and differentiability properties of functions, Princeton Math. Series 30, Princeton University Press, Princeton, NJ, 1970. MR 44 #7280 Zbl 0207.13501 270 SILVANO DELLADIO

Received August 18, 2005.

SILVANO DELLADIO DIPARTIMENTO DI MATEMATICA UNIVERSITADI` TRENTO VIA SOMMARIVE 14, POVO 38050 TRENTO ITALY [email protected] PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS

STEFAN FRIEDL

Cochran introduced Alexander polynomials over noncommutative Laurent polynomial rings. Their degrees were studied by Cochran, Harvey and Tu- raev, who gave lower bounds on the Thurston norm. We extend Cochran’s deﬁnition to twisted Alexander polynomials, and show how Reidemeister torsion relates to these invariants, giving lower bounds on the Thurston norm in terms of the Reidemeister torsion. This yields a concise formulation of the bounds of Cochran, Harvey and Turaev. The Reidemeister torsion approach also provides a natural approach to proving and extending certain monotonicity results of Cochran and Harvey.

1. Introduction

The following algebraic setup allows us to define twisted noncommutative Alexan- der polynomials. Let ދ be a (skew) field and γ : ދ → ދ a ring homomorphism. ±1 Denote by ދγ [t ] the skew Laurent polynomial ring over ދ, so the elements ±1 Pn i in ދγ [t ] are formal sums i=m ai t (m ≤ n ∈ ޚ) with ai ∈ ދ. Addition is given by addition of the coefficients, and multiplication is defined using the rule ti a = γ i (a)ti for any a ∈ ދ. Let X be a connected CW complex with finitely many cells in dimension i. ±1 ±1 Given a representation α : π1(X) → GL(ދγ [t ], d) we can consider the ދγ [t ]- α ±1 d modules Hi (X; ދγ [t ] ) and we define twisted noncommutative Alexander poly- α ±1 nomials 1i (t) ∈ ދγ [t ] (see Section 3.3 for details). Twisted Alexander polynomials over commutative Laurent polynomial rings were first introduced in [Lin 2001], Alexander polynomials over skew Laurent polynomial rings in [Cochran 2004]. Our definition is a combination of the definitions in [Kirk and Livingston 1999] and [Cochran 2004]. In Theorem 3.1 we describe the indeterminacy of these polynomials. ±1 Denote by ދγ (t) the quotient field of ދγ [t ]. We denote the induced represen- ±1 tation π1(X) → GL(ދγ [t ], d) → GL(ދγ (t), d) by α as well. If the homology α d groups Hi (X; ދγ (t) ) vanish and if X is a finite connected CW complex, then

MSC2000: primary 57M27; secondary 57N10. Keywords: Thurston norm, Reidemeister torsion, 3-manifolds, knot genus.

271 272 STEFAN FRIEDL we can define the Reidemeister torsion τ(X, α) ∈ K1(ދγ (t))/ ± α(π1(X)). An important tool is the Dieudonne´ determinant, which defines an isomorphism × det : K1(ދγ (t)) → ދγ (t)ab, × × where ދγ (t)ab denotes the abelianization of the multiplicative group ދγ (t) = × ދγ (t) \{0}. We can therefore study det τ(X, α) ∈ ދγ (t)ab/ ± det α(π1(X)). We refer to Sections 2.3 and 3.1 for details. The following result generalizes commutative results of Turaev [1986; 2001] and Kirk and Livingston [1999]. Theorem 1.1. Let X be a finite connected CW complex of dimension n. Let α : ±1 α d π1(X) → GL(ދγ [t ], d) be a representation such that H∗ (X; ދγ (t) ) = 0. Then α 1i (t) 6= 0 for all i and n−1 Y α (−1)i+1 × e det τ(X, α) = 1i (t) ∈ ދγ (t)ab/{kt | k ∈ ދ \{0}, e ∈ ޚ}. i=0 Pn i ±1 For f (t) = i=m ai t ∈ ދγ [t ]\{0} with am 6= 0, an 6= 0, we define its degree to be deg f (t) = n−m. We can extend this to a degree function deg : ދγ (t)\{0} → ޚ. We denote deg det τ(X, α) by deg τ(X, α). Theorem 1.1 then implies that the degree of τ(X, α) is the alternating sum of the degrees of the twisted Alexander polynomials (see Corollary 3.6). We now turn to the study of 3-manifolds. Here and throughout the paper we will assume that all manifolds are compact, orientable and connected. Recall that given a 3-manifold M and φ ∈ H 1(M; ޚ) the Thurston norm [1986] of φ is defined as ˆ kφkT = min{−χ(S) | S ⊂ M properly embedded surface dual to φ} where Sˆ denotes the result of discarding all connected components of S with positive Euler characteristic. As an example, consider X (K ) = S3 \ νK , where K ⊂ S3 is a knot and νK denotes an open tubular neighborhood of K in S3. Let 1 φ ∈ H (X (K ); ޚ) be a generator, then kφkT = 2 genus(K ) − 1. Let X be a connected CW complex and let φ ∈ H 1(X; ޚ). We identify hence- 1 forth H (X; ޚ) with Hom(H1(X; ޚ), ޚ) and Hom(π1(X), ޚ). A representation ±1 α : π1(X) → GL(ދγ [t ], d) is called φ-compatible if for any g ∈ π1(X) we have α(g) = Atφ(g) for some A ∈ GL(ދ, d). This generalizes a notion of Turaev [2002a]. The following theorem gives lower bounds on the Thurston norm using Reide- meister torsion. It contains the lower bounds of McMullen [2002], Cochran [2004], Harvey [2005], Turaev [2002a] and of the paper [Friedl and Kim 2006]. To our knowledge this theorem is the strongest of its kind. Not only does it contain these results, the formulation of the inequalities given in the papers just cited in terms REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 273 of the degrees of Reidemeister torsion also gives a very concise reformulation of their results. Theorem 1.2. Let M be a 3-manifold with empty or toroidal boundary. Let φ ∈ 1 ±1 H (M; ޚ) and α : π1(M) → GL(ދγ [t ], d) a φ-compatible representation. Then α 6= τ(M, α) is defined if and only if 11 (t) 0. If τ(M, α) is defined, then 1 kφk ≥ deg τ(M, α). T d If (M, φ) fibers over S1, then n 1 o kφk = max 0, deg τ(M, α) . T d The most commonly used skew fields are the quotient fields ދ(G) of group rings ކ[G] (ކ a commutative field) for certain torsion-free groups G, we refer to Section 5.1 for details. The following theorem says roughly that larger groups give better bounds on the Thurston norm. The main idea of the proof is to use the fact that Reidemeister torsion behaves well under ring homomorphisms, in contrast to Alexander polynomials. See Section 6 or [Harvey 2006] for the definition of an admissible triple. Theorem 1.3. Let M be a 3-manifold with empty or toroidal boundary or let M 1 be a 2-complex with χ(M) = 0. Let φ ∈ H (M; ޚ). Let α : π1(M) → GL(ކ, d), ކ a commutative field, be a representation and (ϕG : π → G, ϕH : π → H, φ) an admissible triple for π1(M), in particular we have epimorphisms G → H → ޚ. Write G0 = Ker{G → ޚ} and H 0 = Ker{H → ޚ}. 0 0 If τ(M, ϕH ⊗α) ∈ K1(ދ(H )(t)) is defined, then τ(M, ϕG ⊗α) ∈ K1(ދ(G )(t)) is defined. Furthermore in that case

deg τ(M, ϕG ⊗ α) ≥ deg τ(M, ϕH ⊗ α). A similar theorem holds for 2-complexes with Euler characteristic zero. As a special case consider the case that α is the trivial representation. Using Theorem 1.1 we can recover the monotonicity results of [Cochran 2004; Harvey 2006]. We hope that our alternative proof using Reidemeister torsion will contribute to the understanding of their results. In the next section we recall the deﬁnition of Reidemeister torsion. In Section 3 we introduce twisted noncommutative Alexander polynomials, compute their indeterminacies in Theorem 3.1 and prove Theorem 1.1. Beginning with Section 4 we concentrate on 3-manifolds: Section 4 gives the proof of Theorem 1.2, Section 5 contains examples of φ-compatible representations, in Section 6 we prove Theorem 1.3, and in Section 7 we show that it implies Cochran’s and Harvey’s monotonicity results. We conclude with a few open questions in Section 8. 274 STEFAN FRIEDL

2. Reidemeister torsion

2.1. Definition of K1(R). For the remainder of the paper we will only consider r associative rings R with 1 6= 0 and with the property that if r 6= s ∈ ގ0, then R is not isomorphic to Rs as an R-module. For such a ring R define GL(R) = −→lim GL(R, d), where the maps GL(R, d) → + 7→ A 0 GL(R, d 1) in the direct system are given by A 0 1 . Then K1(R) is defined as GL(R)/[GL(R), GL(R)]. Note that K1(R) is an abelian group. For details we refer to [Milnor 1966; Turaev 2001]. There exists a canonical map GL(R, d) → K1(R) for every d, in particular there exists a homomorphism from the units of R into K1(R). By abuse of notation we denote the image of A ∈ GL(R, d) in K1(R) by A as well. We denote by −A the product of A ∈ K1(R) by the image of (−1) under the map GL(R, 1) → K1(R). We will often make use of the observation (see [Rosenberg 1994, p. 61]) that for A ∈ GL(R, d1), B ∈ GL(R, d2) the product AB ∈ K1(R) is given by A 0 AB = ∈ K (R). 0 B 1

2.2. Definition of Reidemeister torsion. Let C∗ be a finite free chain complex of R-modules. By this we mean a chain complex of free finite right R-modules such that Ci = 0 for all but finitely many i ∈ ޚ. Let Ꮿi ⊂ Ci be a basis for all i with Ci 6= 0. Assume that Bi = Im(Ci+1) ⊂ Ci is free, pick a basis Ꮾi of Bi and a lift ˜ ˜ Ꮾi of Ꮾi to Ci+1. We write Ꮾi Ꮾi−1 for the collection of elements given by Ꮾi ˜ and Ꮾi−1. Since C∗ is acyclic this is indeed a basis for Ci . Then we define the Reidemeister torsion of the based acyclic complex (C∗, {Ꮿi }) to be Y ˜ (−1)i+1 τ(C∗, {Ꮿi }) = [Ꮾi Ꮾi−1/Ꮿi ] ∈ K1(R), where [d/e] denotes the matrix of a basis change, i.e. [d/e] = (ai j ) where di = P j a ji e j . (In contrast to Turaev’s book we view vectors as column vectors, so our matrix is the transpose of the matrix in [Turaev 2001, p. 1].) It is easy to see that τ(C∗, {Ꮿi }) is independent of the choice of {Ꮾi } and of the ˜ choice of the lifts Ꮾi . If the R-modules Bi are not free, then one can show that they are stably free and a stable basis will then make the definition work again. See [Milnor 1966, p. 369] or [Turaev 2001, p. 13] for details.

2.3. Reidemeister torsion of a CW complex. Let X be a connected CW complex. Denote the universal cover of X by X˜ . We view C∗(X˜ ), the chain complex of the universal cover, as a chain complex of right ޚ[π1(X)]-modules, where the ޚ[π1(X)]-module structure is given via deck transformations. Let R be a ring. Let α : π1(X) → GL(R, d) be a representation. This equips d R with a left ޚ[π1(X)]-module structure. We can therefore consider the chain REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 275

α ; d = ˜ ⊗ d complex C∗ (X R ) C∗(X) ޚ[π1(X)] R . This is a ﬁnite free chain complex of α d (right) R-modules. We denote its homology by H∗ (X; R ), we drop α from the notation if it is clear from the context. Now assume that X is a ﬁnite connected CW complex. If

α d α d Hi (X; R ) = Hi (C∗ (X; R )) 6= 0

1 ri for some i, we write τ(X, α) = 0. Otherwise denote the i-cells of X by σi , . . . , σi d and denote by e1,..., ed the standard basis of R . Pick an orientation for each cell j j j ˜ σi , and also pick a lift σ˜i for each cell σi to the universal cover X. We get a basis

1 1 ri ri Ꮿi = {σ ˜i ⊗ e1,..., σ˜i ⊗ ed ,..., σ˜i ⊗ e1,..., σ˜i ⊗ ed } α d for Ci (X; R ). Then we can deﬁne α d τ(C∗ (X; R ), {Ꮿi }) ∈ K1(R). This element depends only on the ordering and orientation of the cells and on the choice of lifts of the cells to the universal cover. Therefore

α d τ(X, α) = τ(C∗ (X, R ), {Ꮿi }) ∈ K1(R)/ ± α(π1(X)) is a well defined invariant of the CW complex X. Now let M be a compact PL manifold. Pick any finite CW structure for M to define τ(M, α) ∈ K1(R)/±α(π1(M)). By Chapman’s theorem [1974] this is a well defined invariant of the manifold (independent of the choice of the CW structure).

2.4. Computation of Reidemeister torsion. We explain an algorithm for computing Reidemeister torsion formulated in [Turaev 2001, Section 2.1] in the commutative case. Assume that we have a ﬁnite free chain complex of R-modules

∂m ∂1 0 → Cm −→ Cm−1 → · · · → C1 −→ C0 → 0. i Let Ai = (a jk) be the matrix representing ∂i corresponding to the given bases. Again, in contrast with Turaev, we view the elements in Rrank(Ci ) as column vectors. Following [Turaev 2001, p. 8] we deﬁne a matrix chain for C to be a collection of sets ξ = (ξ0, ξ1, . . . , ξm) where ξi ⊂ {1, 2,..., rank(Ci )} so that ξ0 = ∅. Given a matrix chain ξ we deﬁne Ai (ξ), i = 1,..., m to be the matrix formed by the i i entries a jk with j 6∈ ξi−1 and k ∈ ξi . Put differently the matrix (a jk) jk is given by considering only the ξi -columns of Ai and with the ξi−1-rows removed. We say that a matrix chain ξ is a τ-chain if A1(ξ), . . . , Am(ξ) are square matrices. The following is the generalization of Turaev’s Theorem 2.2 to the noncommutative setting. His proof can easily be generalized to this more general setting. 276 STEFAN FRIEDL

Theorem 2.1. Let ξ be a τ-chain such that Ai (ξ) is invertible for all odd i. Then Ai (ξ) is invertible for all even i if and only if H∗(C) = 0. If H∗(C) = 0, then m Y (−1)i τ(C) = Ai (ξ) ∈ K1(R) for some ∈ {±1}. i=1 This proposition is the reason why Reidemeister torsion behaves in general well under ring homomorphisms.

3. Reidemeister torsion and Alexander polynomials

3.1. Laurent polynomial rings and the Dieudonné determinant. For the remain- ±1 der of this paper let ދ be a (skew) field and let ދγ [t ] be a skew Laurent polyno- ±1 mial ring. By [Dodziuk et al. 2003, Corollary 6.3] the ring ދγ [t ] has a classical ±1 quotient field ދγ (t) which is flat over ދγ [t ] (compare [Ranicki 1998, p. 99]). In ±1 particular we can view ދγ [t ] as a subring of ދγ (t) and any element in ދγ (t) is −1 ±1 ±1 of the form f (t)g(t) for some f (t) ∈ ދγ [t ] and g(t) ∈ ދγ [t ]\{0}. We refer × to Theorem 5.1 for a related result. Recall that we write ދγ (t) = ދγ (t) \{0}. In the following we mean by an elementary column (row) operation the addition of a right multiple (left multiple) of one column (row) to a different column (row). Let A be an invertible k × k matrix over the skew field ދγ (t). After elementary row operations we can turn A into a diagonal matrix D = (di j ). Then the × × × × Dieudonne´ determinant det A ∈ ދγ (t)ab = ދγ (t) /[ދγ (t) , ދγ (t) ] is defined Qk to be i=1 dii . This is a well defined map. The Dieudonne´ determinant is invariant under elementary row and column operations, and induces an isomorphism × det : K1(ދγ (t)) → ދγ (t)ab. Using the last observation in Section 2.1 it is easy to see that A = det A ∈ K1(ދγ (t)). We will often make use of this equality. We refer to [Rosenberg 1994, Theorem 2.2.5 and Corollary 2.2.6] for more details. ±1 In the introduction we defined deg : ދγ [t ]\{0} → ގ. This can be extended to × −1 a homomorphism deg : ދγ (t) → ޚ via deg( f (t)g(t) ) = deg f (t)−deg g(t) for ±1 × × f (t), g(t) ∈ ދγ [t ]\{0}. Clearly the degree map vanishes on [ދγ (t) , ދγ (t) ] × and we get an induced homomorphism K1(ދγ (t)) → ދγ (t)ab → ޚ which we also denote by deg.

±1 ±1 3.2. Orders of ދγ [t ]-modules. Let H be a finitely generated right ދγ [t ]- ±1 module. The ring ދγ [t ] is a principal ideal domain (PID) since ދ is a skew field. We can therefore find an isomorphism l ∼ M ±1 ±1 H = ދγ [t ]/pi (t)ދγ [t ] i=1 ±1 for pi (t)∈ ދγ [t ] for i =1,..., l. Following [Cochran 2004] we define ord(H)= Ql ±1 i=1 pi (t) ∈ ދγ [t ]. This is called the order of H. REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 277

±1 Note that ord(H) ∈ ދγ [t ] has a high degree of indeterminacy. For example writing the pi (t) in a different order will change ord(H). Furthermore we can e × change pi (t) by multiplication by any element of the form kt where k ∈ ދ = ދ \{0} and e ∈ ޚ. The following theorem can be viewed as saying that these are all possible indeterminacies. ±1 Theorem 3.1. Let H be a finitely generated right ދγ [t ]-module. Then ord(H) = ±1 ±1 0 if and only if H is not ދγ [t ]-torsion. If ord(H) 6= 0, then ord(H) ∈ ދγ [t ] × is well defined considered as an element in ދγ (t)ab up to multiplication by an element of the form kte, k ∈ ދ× and e ∈ ޚ. The first statement is clear. We postpone the proof of the second statement of the theorem to Section 3.4. We refer to [Cochran 2004, p. 367] for an alternative discussion of the indeterminacy of ord(H); the idea of considering ord(H) as an × element in ދγ (t)ab is already present there. It follows from Theorem 3.1 that deg ord(H) is well defined. In fact we have the following interpretation of ord(H). ±1 Lemma 3.2 [Cochran 2004, p. 368]. Let H be a finitely generated right ދγ [t ]- torsion module. Then deg ord(H) = dimދ H. Here we used that by [Stenstrom¨ 1975, Proposition I.2.3] and [Cohn 1985, p. 48] every right ދ-module V is free and has a well defined dimension dimދ(V ). ±1 Proof. It is easy to see that for f (t) ∈ ދγ [t ]\{0} we have ±1 ±1 deg f (t) = dimދ(ދγ [t ]/f (t)ދγ [t ]).

The lemma is now immediate. 3.3. Alexander polynomials. Let X be a connected CW complex with finitely ±1 many cells in dimension i. Let α : π1(X) → GL(ދγ [t ], d) be a representation. ±1 ±1 d The right ދγ [t ]-module Hi (X; ދγ [t ] ) is called twisted (noncommutative) Alexander module. Similar modules were studied in [Cochran 2004; Harvey 2005; ±1 d ±1 Turaev 2002a]. Note that Hi (X; ދγ [t ] ) is a finitely generated ދγ [t ]-module since we assumed that X has only finitely many cells in dimension i and since ±1 α ±1 d ±1 ދγ [t ] is a PID. We now define 1i (t) = ord(Hi (X; ދγ [t ] )) ∈ ދγ [t ], this is called the (twisted) i-th Alexander polynomial of (X, α). The degrees of these polynomials (corresponding to one-dimensional representations) have been studied recently in various contexts [Cochran 2004; Har- vey 2005; 2006; Turaev 2002a; Leidy and Maxim 2006; Friedl and Kim 2005; Friedl and Harvey 2006]. We hope that by determining the indeterminacy of the Alexander polynomials (Theorem 3.1) more information can be extracted from the Alexander polynomials than just the degrees. 278 STEFAN FRIEDL

±1 3.4. Proof of Theorem 3.1. We first point out that ދγ [t ] is a Euclidean ring with ±1 respect to the degree function. This means that given f (t), g(t) ∈ ދγ [t ]\{0} we ±1 can find a(t), r(t) ∈ ދγ [t ] such that f (t) = g(t)a(t) + r(t) and such that either r(t) = 0 or deg r(t) < deg g(t). ±1 Let A be an r × s matrix over ދγ [t ] of rank r. Here and in the following ±1 the rank of a matrix over ދγ [t ] will be understood as the rank of the matrix considered as a matrix over the skew field ދγ (t). Note that rank(A) = r implies ±1 that in particular s ≥ r. Since ދγ [t ] is a Euclidean ring we can perform a sequence of elementary row and column operations to turn A into a matrix of the form D 0r×(s−r) where D is an r ×r matrix and 0r×(s−r) stands for the r ×(s−r) matrix consisting only of zeros. Since A is of rank r it follows that D has rank r as well, in particular D is a square matrix which is invertible over ދγ (t) and we can × consider its Dieudonne´ determinant det D. We define det A = det D ∈ ދγ (t)ab. ±1 Lemma 3.3. Let A be a (square) matrix over ދγ [t ] which is invertible over ދγ (t). × (1) The Dieudonné determinant det A ∈ ދγ (t)ab can be represented by an element ±1 in ދγ [t ]\{0}. ±1 × (2) If A ∈ GL(ދγ [t ], d), then det A ∈ ދγ (t)ab can be represented by an element of the form kte, k ∈ ދ×, e ∈ ޚ. (3) The Dieudonné determinant induces a homomorphism × det : K1(ދγ (t)) → ދγ (t)ab ±1 e × × × × sending K1(ދγ [t ]) to {kt |k ∈ ދ , e ∈ ޚ}/[ދγ (t) , ދγ (t) ] ⊂ ދγ (t)ab. Proof. The first statement follows from the discussion preceding the lemma. Now ±1 let A ∈ GL(ދγ [t ], r). Then det A has degree zero, by Lemma 3.2 applied to ±1 r ±1 r H = ދγ [t ] /Aދγ [t ] . This proves the second statement. The last statement follows from the second statement and the fact that the Dieudonne´ determinant × induces a homomorphism det : K1(ދγ (t)) → ދγ (t)ab. ±1 Proposition 3.4. Let A be an r × s matrix over ދγ [t ] of rank r. Then det A ∈ × e ދγ (t)ab is well defined up to multiplication by an element of the form kt , k ∈ ދ×, e ∈ ޚ. Furthermore det A is invariant under elementary row and column operations. ±1 Proof. First note that the effect of an elementary row operation on A over ދγ [t ] ±1 can be described by left multiplication by a matrix P ∈ GL(ދγ [t ], r). Similarly ±1 an elementary column operation on A over ދγ [t ] can be described by right ±1 multiplication by an s × s matrix Q ∈ GL(ދγ [t ], s). ±1 ±1 Now assume we have P1, P2 ∈ GL(ދγ [t ], r) and Q1, Q2 ∈ GL(ދγ [t ], s) such that Pi AQi = Di 0r×(s−r) , i = 1, 2 where Di is an r × r matrix. We are REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 279

e × × done once we show that det D1 = kt det D2 ∈ ދγ (t)ab for some k ∈ ދ , e ∈ ޚ. −1 Let Ei = Pi Di . Then by Lemma 3.3 we only have to show that E1 = E2 ∈ ±1 K1(ދγ (t))/K1(ދγ [t ]). −1 = −1 := −1 ∈ [ ±1] We have E1 0 Q1 E2 0 Q2 . Let Q Q2 Q1 GL(ދγ t , s), we therefore get the equality E1 0 = E2 0 Q. Now write

Q Q Q = 11 12 , Q21 Q22

±1 where Qi j is a ni × n j matrix over ދγ [t ] with n1 = r and n2 = s − r. We get the equality E 0 E 0 Q Q 1 = 2 11 12 . Q21 Q22 0 ids−r Q21 Q22

It follows in particular that Q22 is invertible over ދγ (t). Furthermore we have

±1 E1 · Q22 = E2 ∈ K1(ދγ (t))/K1(ދγ [t ]).

±1 Note that deg : K1(ދγ (t)) → ޚ vanishes on K1(ދγ [t ]) by Lemma 3.3. We therefore get deg det E1+deg det Q22 =deg det E2, in particular deg det E1 ≤deg det E2. But by symmetry we have deg det E2 ≤ deg det E1. In particular deg det Q22 = 0. The proposition now follows immediately from Lemma 3.3 since deg f (t) = 0 for ±1 e × f (t) ∈ ދγ [t ]\{0} if and only if f (t) = kt for some k ∈ ދ , e ∈ ޚ. The last statement is immediate. ±1 Let H be a ﬁnitely generated right ދγ [t ]-module. We say that an r ×s matrix A is a presentation matrix for H if the following sequence is exact:

±1 s A ±1 r ދγ [t ] −→ ދγ [t ] → H → 0. We say that A has full rank if the rank of A equals r. Note that A has full rank if

⊗ ±1 = and only if H ދγ [t ] ދγ (t) 0. The following lemma clearly implies Theorem 3.1. ±1 Lemma 3.5. Let H be a ﬁnitely generated right ދγ [t ]-module and let A1, A2 be presentation matrices for H. Then A1 has full rank if and only if A2 has full rank. Furthermore if Ai has full rank, then

× e det A1 = det A2 ∈ ދγ (t)ab/{kt |k ∈ ދ \{0}, e ∈ ޚ}. Proof. Two presentation matrices for H differ by a sequence of matrix moves of the following forms and their inverses: (1) Permutation of rows or columns. A 0 (2) Replacement of the matrix A by . 0 1 280 STEFAN FRIEDL

(3) Addition of an extra column of zeros to the matrix A. (4) Addition of a right scalar multiple of a column to another column. (5) Addition of a left scalar multiple of a row to another row. This result coincides with [Lickorish 1997, Theorem 6.1] in the commutative case, ±1 but the proof there carries through in the case of the base ring ދγ [t ] as well (compare [Harvey 2005, Lemma 9.2]). Clearly none of the moves changes the status of being of full rank, and if a representation is of full rank, then it is easy to see that none of the moves changes the determinant. 3.5. Proof of Theorem 1.1. Now let X be a finite connected CW complex of ±1 dimension n. Let α : π1(X) → GL(ދγ [t ], d) be a representation such that α d H∗ (X; ދγ (t) ) = 0. (Recall that we denote the induced representation π1(X) → ±1 GL(ދγ (t), d) by α as well). Furthermore recall that ދγ (t) is flat over ދγ [t ], d ±1 d ; = ; [ ] ⊗ ±1 in particular Hi (X ދγ (t) ) Hi (X ދγ t ) ދγ [t ] ދγ (t). It follows that d ±1 d ±1 Hi (X; ދγ (t) ) = 0 if and only if Hi (X; ދγ [t ] ) is ދγ [t ]-torsion, which is α equivalent to 1i (t) 6= 0. This proves the first statement of Theorem 1.1. To conclude the proof of Theorem 1.1 it remains to prove the following claim. α d Claim. If H∗ (X; ދγ (t) ) = 0, then

n−1 Y α (−1)i+1 × e det τ(X, α) = 1i (t) ∈ ދγ (t)ab/{kt |k ∈ ދ \{0}, e ∈ ޚ}. i=0 = ˜ ⊗ [ ±1]d [ ±1] Proof. Let C∗ C∗(X) ޚ[π1(X)] ދγ t . Any ދγ t -basis for C∗ also gives a ⊗ ±1 basis for C∗ ދγ [t ] ދγ (t), which we will always denote by the same symbol. ±1 Denote by Ꮿ∗ the ދγ [t ]-basis of C∗ as in Section 2.3. Let

:= ⊗ ±1 ri dimދ(t)(Ci ދγ [t ] ދγ (t)),

∂i := { ⊗ ±1 −→ ⊗ ±1 } si dimދ(t)(Ker Ci ދγ [t ] ދγ (t) Ci−1 ދγ [t ] ދγ (t) ).

d ∂i Note that si + si−1 = ri since H∗(X; ދγ (t) ) = 0. Note also that Ker{Ci −→ Ci−1} ±1 is a free direct summand of Ci of rank si since ދγ [t ] is a PID. We can therefore [ ±1] 0 = { } { } pick ދγ t -bases Ꮿi v1, . . . , vri for Ci such that v1, . . . , vsi is a basis for 0 the kernel in question. Base changes from Ꮿi to Ꮿi are given by matrices which ±1 are invertible over ދγ [t ], so 0 ±1 ⊗ ±1 { } = ⊗ ±1 { } ∈ [ ] τ(C∗ ދγ [t ]ދγ (t), Ꮿi ) τ(C∗ ދγ [t ]ދγ (t), Ꮿi ) K1(ދγ (t))/K1(ދγ t ).

Let Ai be the ri−1 ×ri matrix representing ∂i : Ci → Ci−1 with respect to the bases 0 0 := { + } = := Ꮿi and Ꮿi−1. Let ξi si 1,..., ri , i 1,..., n and ξ0 ∅. Let Ai (ξ) as in REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 281

±1 Theorem 2.1. Note that Ai (ξ) is an si−1 × si−1 matrix over ދγ [t ]. In particular ξ := (ξ0, . . . , ξn) is a τ-chain. It is easy to see that 0si−1×si Ai (ξ) Ai = . 0si−2×si 0si−2×si−1

Since Ai has rank si−1 it follows that Ai (ξ) is invertible over ދγ (t). It follows from Theorem 2.1 that n 0 Y (−1)i ⊗ ±1 { } = ∈ τ(C∗ ދγ [t ] ދγ (t), Ꮿi ) Ai (ξ) K1(ދγ (t)). i=1 We also have short exact sequences

A (ξ) ±1 si−1 i ±1 si−1 ±1 d 0 → ދγ [t ] −−−→ ދγ [t ] → Hi−1(C∗) = Hi−1(X; ދγ [t ] ) → 0.

±1 d In particular (Ai (ξ)) is a presentation matrix for Hi−1(X; ދγ [t ] ). It therefore = α follows from Lemma 3.5 that det Ai (ξ) 1i−1(t). × The following corollary now follows from the fact that deg : ދγ (t) → ޚ is a homomorphism and from Lemma 3.2. Corollary 3.6. Let X be a ﬁnite connected CW complex of dimension n. Let ±1 α d α : π1(X) → GL(ދγ [t ], d) be a representation such that H∗ (X; ދγ (t) ) = 0. Then n−1 n−1 X i+1 α X i+1 ±1 d deg τ(X, α) = (−1) deg 1i (t) = (−1) dimދ(Hi (X; ދγ [t ] ). i=0 i=0

d ±1 Remark. In the case that H∗(X; ދγ (t) ) 6= 0 we can pick ދγ [t ]-bases Ᏼi for ±1 ±1 d d the ދγ [t ]-free parts of Hi (X; ދγ [t ] ). These give bases for Hi (X; ދγ (t) ) = ±1 d ; [ ] ⊗ ±1 Hi (X ދγ t ) ދγ [t ] ދγ (t) and we can consider { } = ˜ ⊗ d { } ∈ τ(X, α, Ᏼi ) τ(C∗(X) ޚ[π1(X)] ދγ (t) , Ᏼi ) K1(ދγ (t))

±1 (see [Milnor 1966] for details). As an element of K1(ދγ (t))/K1(ދγ [t ]), this τ(X, α, {Ᏼi }) is independent of the choice of {Ᏼi }. The proof of Theorem 1.1 can be generalized to show that it is the alternating product of the orders of the ±1 ±1 d ދγ [t ]-torsion submodules of H∗(X; ދγ [t ] ) (compare [Kirk and Livingston 1999] in the commutative case).

4. 3-manifolds and 2-complexes

We now restrict ourselves to φ-compatible representations since these have a closer connection to the topology of a space. 282 STEFAN FRIEDL

Lemma 4.1. Let X be a connected CW complex with ﬁnitely many cells in dimen- 1 ±1 sions 0 and 1. Let φ ∈ H (X; ޚ) nontrivial and let α : π1(X) → GL(ދγ [t ], d) α 6= be a φ-compatible representation. Then 10 (t) 0. If X is in fact an k-manifold, α then 1k (t) = 1.

We need the following notation. If A = (ai j ) is an r × s matrix over ޚ[π1(X)] and α : π1(X) → GL(R, d) a representation. Then we denote by α(A) the rd ×sd matrix over R obtained by replacing each entry ai j ∈ ޚ[π1(X)] of A by the d × d matrix α(ai j ).

Proof. First equip X with a CW structure with one 0-cell and n 1-cells g1,..., gn. We denote the corresponding elements in π1(X) by g1,..., gn as well. Since φ is 6= = ˜ ⊗ nontrivial there exists at least one i such that φ(gi ) 0. Write C∗ C∗(X) ޚ[π1(X)] ±1 d ދγ [t ] . The boundary map ∂1 : C1 → C0 is represented by the matrix

(α(1 − g1), . . . , α(1 − gn)) = (id − α(g1), . . . , id − α(gn)).

φ(g ) Since α is φ-compatible it follows that α(1 − gi ) = id − At i for some matrix A ∈ GL(ދ, d). The ﬁrst statement of the lemma now follows from Lemma 4.2. If X is a closed k-manifold then equip X with a CW structure with one k-cell. Since φ is primitive and φ-compatible an argument as above shows that ∂k : Ck → ±1 d α Ck−1 has full rank, i.e. Hk(X; ދγ [t ] ) = 0. Hence 1k (t) = 1. If X is a k- manifold with boundary, then it is homotopy equivalent to a (k−1)-complex, and ±1 d hence Hk(X; ދγ [t ] ) = 0. ±1 Lemma 4.2. Let ދγ [t ] be a skew Laurent polynomial ring and let A, B be invertible d × d matrices over ދ and r 6= 0. Then deg det(A + Btr ) = dr. In r particular A + Bt is invertible over ދγ (t). Harvey has proved a related result [2005, Proposition 9.1]. d Proof. We can clearly assume that r > 0. Let {e1,..., ed } be a basis for ދ . ±1 d ±1 d r ±1 d Consider the projection map p :ދγ [t ] → P =ދγ [t ] /(A+Bt )ދγ [t ] . By j Lemma 3.2 we are done once we show that p(ei t ), i ∈ {1,..., d}, j ∈ {0,..., r − 1} form a basis for P as a right ދ-vector space. It follows easily from the fact that A, B are invertible that this is indeed a gen- ±1 d Pm i d erating set. Let v ∈ ދγ [t ] \{0}. We can write v = i=n vi t , vi ∈ ދ with r vn 6= 0, vm 6= 0. Since A, B are invertible it follows that (A+ Bt )v has terms with t-exponent n and terms with t-exponent m + r. This observation can be used to show that the vectors in question are linearly independent in P. We can now give the proof of Theorem 1.2. Proof of Theorem 1.2. Now let M be a 3-manifold whose boundary is empty or consists of tori. A standard duality argument shows that 2χ(M) = χ(∂ M) = 0. REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 283

1 ±1 Let φ ∈ H (M; ޚ) be nontrivial, and α : π1(M) → GL(ދγ [t ], d) a φ-compatible representation. ; d = α 6= We ﬁrst show that H∗(M ދγ (t) ) 0 if and only if 11 (t) 0. In Sec- d α tion we showed that Hi (M; ދγ (t) ) = 0 if and only if 1i (t) 6= 0. It now fol- ; d = = α 6= lows from Lemma 4.1 that Hi (M ދγ (t) ) 0 for i 0, 3. If 11 (t) 0, then d d H1(M; ދγ (t) ) = 0. Since χ(Hi (M; ދγ (t) )) = dχ(M) = 0 it follows that d H2(M; ދγ (t) ) = 0.

Claim. kφkT is bounded below by 1 − α ; [ ±1]d + α ; [ ±1]d − α ; [ ±1]d d dimދ H0 (M ދγ t ) dimދ H1 (M ދγ t ) dimދ H2 (M ދγ t ) . This inequality becomes an equality if (M, φ) ﬁbers over S1 and if M 6= S1 × D2, M 6= S1 × S2.

Proof. If φ vanishes on X ⊂ M, the restriction of α to π1(X) lies in GL(ދ, d) ⊂ ±1 GL(ދγ [t ], d) since α is φ-compatible. Therefore

α ±1 d ∼ α d ±1 Hi (X; ދγ [t ] ) = Hi (X; ދ ) ⊗ދ ދγ [t ]. The proofs of [Friedl and Kim 2006, Theorem 3.1 and Theorem 6.1] can now easily be translated to this noncommutative setting. This proves the claim. Combining the results of the claim with Lemma 3.2 and Corollary 3.6 we immediately get a proof for Theorem 1.2. In order to relate Theorem 1.2 to the results of [Cochran 2004; Harvey 2005; Tu- raev 2002a] we need the following computations for one-dimensional φ-compatible representations. Recall that φ ∈ H 1(X; ޚ) is called primitive if the corresponding map φ : H1(X; ޚ) → ޚ is surjective. Lemma 4.3. Let X be a connected CW complex with ﬁnitely many cells in dimen- 1 ±1 sions zero and one. Let φ ∈ H (X; ޚ) primitive. Let α : π1(X) → GL(ދγ [t ], 1) ±1 be a φ-compatible one-dimensional representation. If α(π1(X)) ⊂ GL(ދγ [t ], 1) α = − ∈ \{ } α = is cyclic, then 10 (t) at 1 for some a ދ 0 . Otherwise 10 (t) 1. Proof. Equip X with a CW structure with one 0-cell and then consider the chain complex for X as in Lemma 4.1. The lemma now follows easily from the obser- ±1 vation that in ދγ [t ] we have gcd(1−at, 1−bt) = 1 if a 6= b ∈ ދ. Lemma 4.4. Let X be a 3-manifold with empty or toroidal boundary or let X be 1 a 2-complex with χ(X) = 0. Let φ ∈ H (M; ޚ) nontrivial. Let α : π1(M) → ±1 GL(ދγ [t ], 1) be a φ-compatible one-dimensional representation. Assume that α 6= α = α 11 (t) 0. If X is a closed 3-manifold, then deg 12 (t) deg 10 (t), otherwise α = 12 (t) 1. 284 STEFAN FRIEDL

Proof. First assume that X is a 3-manifold. Then the lemma follows from combining [Turaev 2002a, Sections 4.3 and 4.4] with [Friedl and Kim 2006, Lem- mas 4.7 and 4.9]. (The latter results also hold in the noncommutative setting.) If X is a 2-complex then the argument in the proof of Theorem 1.2 shows that d ±1 d H2(X; ދγ (t) ) = 0. But since X is a 2-complex we have H2(X; ދγ [t ] ) ⊂ ; d ; [ ±1]d = α = H2(X ދγ (t) ), hence H2(X ދγ t ) 0 and 12 (t) 1. Remark. We cannot apply the duality results of [Friedl and Kim 2006, Lemma 4.12 and Proposition 4.13] since the natural involution on ޚ[G] does not necessarily ±1 ±1 extend to an involution on ދγ [t ], i.e., the representation ޚ[G] → ދγ [t ] is not necessarily unitary. It now follows immediately from Lemma 4.3 and 4.4 and the discussion in Sec- tion 5 that Theorem 1.2 contains the results of [McMullen 2002; Cochran 2004; Harvey 2005; Turaev 2002a; Friedl and Kim 2006]. Remark. Given a 2-complex X, Turaev [2002b] deﬁned a norm

1 k kX : H (X; ޒ) → ޒ, modeled on the deﬁnition of the Thurston norm of a 3–manifold. He then gave lower bounds for the Turaev norm (see also [Turaev 2002a]) which have the same form as certain lower bounds for the Thurston norm. Going through the proofs in [Friedl and Kim 2006] it is not hard to see that the obvious version of Theorem 1.2 for 2-complexes also holds. If M is a 3-manifold with boundary, then it is homotopy equivalent to a 2- complex X. It is not known whether the Thurston norm of M agrees with the Turaev norm on X. But the fact that Theorem 1.2 holds in both cases, and the observation that deg τ(X, α) is a homotopy invariant by Theorem 1.1 suggests that they do in fact agree.

5. Examples for skew ﬁelds and φ-compatible representations

5.1. Skew fields of group rings. A group G is called locally indicable if for every finitely generated subgroup U ⊂ G there exists a nontrivial homomorphism U → ޚ. Theorem 5.1. Let G be a locally indicable and amenable group and let ކ be a commutative field. Then the following hold. (1) ކ[G] is an Ore domain, in particular it embeds in its classical right ring of quotients ދ(G). (2) ދ(G) is flat over ކ[G]. It follows from [Higman 1940] that ކ[G] has no zero divisors. The first part now follows from [Tamari 1957] or [Dodziuk et al. 2003, Corollary 6.3]. Part (b) REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 285 is a property of Ore localizations (see [Ranicki 1998, p. 99], for example). We call ދ(G) the Ore localization of ކ[G]. A group G is called poly–torsion-free–abelian (PTFA) if there exists a filtration

1 = G0 ⊂ G1 ⊂ · · · ⊂ Gn−1 ⊂ Gn = G such that Gi /Gi−1 is torsion free abelian. PTFA groups are amenable and locally indicable [Strebel 1974]. The group rings of PTFA groups played an important role in [Cochran et al. 2003; Cochran 2004; Harvey 2005]. 5.2. Examples for φ-compatible representations. Let X be a connected CW complex and φ ∈ H 1(X; ޚ). We give examples of φ-compatible representations. 1 ∼ Let ކ be a commutative field. The element φ ∈ H (X; ޚ) = Hom(H1(X; ޚ), hti) ±1 induces a φ-compatible representation φ : ޚ[π1(X)] → ކ[t ]. Furthermore if ±1 α : π1(X) → GL(ކ, d) is a representation, then π1(X) acts via α ⊗φ on the ކ[t ]- d ±1 ∼ ±1 d module ކ ⊗ކ ކ[t ] = ކ[t ] . We therefore get a representation α⊗φ : π1(X) → GL(ކ[t±1], d), which is clearly φ-compatible. In this particular case Theorem 1.2 was proved in [Friedl and Kim 2006]. To describe the φ-compatible representations of Cochran [2004] and Harvey [2005; 2006] we need the following definition. Definition. Let π be a group, φ : π → ޚ an epimorphism and ϕ : π → G an epimorphism to a locally indicable and amenable group G such that there exists a map φG : G → ޚ (which is necessarily unique) such that ϕ π / G @ @@ @@ φG φ @@ @ ޚ commutes. Following [Harvey 2006, Definition 1.4] we call (ϕ, φ) an admissible pair. If φG is an isomorphism, then (ϕ, φ) is called initial.

Now let (ϕ : π1(X) → G, φ) be an admissible pair for π1(X). In the following we denote Ker{φ : G → ޚ} by G0(φ). When the homomorphism φ is understood we will write G0 for G0(φ). Clearly G0 is still a locally indicable and amenable group. Let ކ be any commutative ﬁeld and ދ(G0) the Ore localization of ކ[G]. Pick an element µ ∈ G such that φ(µ) = 1. Let γ : ދ(G0) → ދ(G0) be the homomorphism given by γ (a) = µaµ−1. Then we get a representation 0 ±1 G → GL(ދ(G )γ [t ], 1) g 7→ (gµ−φ(g)tφ(g)).

0 ±1 It is clear that α : π1(X) → G → GL(ދ(G )γ [t ], 1) is φ-compatible. Note 0 ±1 that the ring ދ(G )γ [t ], and hence the representation above, depend on the 286 STEFAN FRIEDL choice of µ. We will nonetheless suppress γ from the notation since different choices of splittings give isomorphic rings. We often make use of the fact that f (t)g(t)−1 → f (µ)g(µ)−1 deﬁnes an isomorphism ދ(G0)(t) → ދ(G) (see [Har- =∼ vey 2005, Proposition 4.5]). Similarly ޚ[G0][t±1] −→ ޚ[G]. An important example of admissible pairs is provided by Harvey’s rational de- (0) rived series of a group G; see [Harvey 2005, Section 3]. Let Gr = G and deﬁne inductively

(n) (n−1) k (n−1) (n−1) Gr = g ∈ Gr | g ∈ Gr , Gr for some k ∈ ޚ \{0} . (n−1) (n) ∼ (n−1) (n−1) (n−1) Note that Gr /Gr = Gr / Gr , Gr /ޚ-torsion. By [Harvey 2005, (n) Corollary 3.6] the quotients G/Gr are PTFA groups for any G and any n. If (n) φ : G → ޚ is an epimorphism, then (G → G/Gr , φ) is an admissible pair for (G, φ) for any n > 0. 3 For example, if K is a knot and G = π1(S \ K ), it follows from [Strebel 1974] (n) (n) that Gr = G , i.e. the rational derived series equals the ordinary derived series (compare [Cochran 2004; Harvey 2005]).

Remark. Recall that for a knot K the knot exterior S3 \ νK is denoted by X (K ). 1 Let π = π1(X (K )) and let φ ∈ H (X (K ); ޚ) primitive. Then

0 0 (n) ±1 δn(K ) = dim 0 0 (n) (H1(X (K ), ދ(π /(π ) )[t ]) ދ(π /(π )r ) r is a knot invariant for n > 0. Cochran [2004, p. 395, Question 5] asked whether K 7→ δn(K ) is of finite type. Eisermann [2000, Lemma 7] has shown that the genus is not a finite type knot invariant. Cochran [2004] showed that δn(K ) ≤ 2 genus(K ) (see also Theorem 1.2 together with Corollary 3.6 and Lemmas 4.3 and 4.4). Eisermann’s argument can now be used to show that K 7→ δn(K ) is not of finite type either. Let X be again be a connected CW complex and φ ∈ H 1(X; ޚ). The two types of φ-compatible representations given above can be combined as follows. Let α : π1(X) → GL(ކ, d) be a representation and let ϕ : π1(X) → G be a homomorphism such that (ϕ, φ) is an admissible pair. Denote the Ore localization of ކ[G0] by 0 0 ±1 d ∼ 0 ±1 d ދ(G ). Then π1(X) acts via ϕ ⊗ α on ދ(G )[t ] ⊗ކ ކ = ދ(G )[t ] . We 0 ±1 therefore get a φ-compatible representation ϕ ⊗ α : π1(X) → GL(ދ(G )[t ], d).

6. Comparing different φ-compatible maps

Deﬁnition [Harvey 2006]. Let π be a group and φ : π → ޚ an epimorphism. Furthermore let ϕ1 : π → G1 and ϕ2 : π → G2 be epimorphisms to locally indicable and amenable groups G1 and G2. We call (ϕ1, ϕ2, φ) an admissible triple for π REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 287

1 : → if there exist epimorphisms ϕ2 G1 G2 (which is not an isomorphism) and : → = 1 ◦ = ◦ φ2 G2 ޚ such that ϕ2 ϕ2 ϕ1, and φ φ2 ϕ2. The situation can be summarized in the diagram

G1 > ϕ1 ~ ~~ 1 ~ ϕ2 ~~ ~~ ϕ2 π / G2 AA AA A φ2 φ AA A ޚ.

In particular, (ϕi , φ), i = 1, 2, are admissible pairs for π. Given an admissible triple we can pick splittings ޚ → Gi of ϕi , i = 1, 2 which make the following diagram commute: ޚ / G A 1 AA AA ϕ1 AA 2 A G2. We therefore get an induced commutative diagram of ring homomorphisms

ޚ[π] / ޚ[G0 ][t±1] J 1 JJ JJ 1 JJ ϕ2 JJ J% [ 0 ][ ±1] ޚ G2 t . (We are suppressing the notation for the twisting in the skew Laurent polynomial 0 ±1 rings.) Denote the φ-compatible maps ޚ[π] → ދ(Gi )[t ], i = 1, 2 by ϕi as well. For convenience we recall Theorem 1.3. Theorem 1.3. Let M be a 3-manifold whose boundary is a (possibly empty) collection of tori or let M be a 2-complex with χ(M) = 0. Let α : π1(M) → GL(ކ, d) be a representation and (ϕ1, ϕ2, φ) an admissible triple for π1(M). If τ(M, ϕ2 ⊗α) 6= 0, then τ(M, ϕ1 ⊗ α) 6= 0. Furthermore in that case

deg τ(M, ϕ1 ⊗ α) ≥ deg τ(M, ϕ2 ⊗ α).

6.1. Proof of Theorem 1.3 for closed 3-manifolds. Let M be a closed 3-manifold. Choose a triangulation of M. Let T be a maximal tree in the 1-skeleton of the triangulation and let T 0 be a maximal tree in the dual 1-skeleton. Following [McMullen 2002, Section 5] we collapse T to form a single 0-cell and join the 3-simplices along T 0 to form a single 3-cell. Since χ(M) = 0 the number n of 1-cells equals 288 STEFAN FRIEDL the number of 2-cells. Consider the chain complex of the universal cover M˜ :

˜ 1 ∂3 ˜ n ∂2 ˜ n ∂1 ˜ 1 0 → C3(M) −→ C2(M) −→ C1(M) −→ C0(M) → 0, where the superscript indicates the rank over ޚ[π1(M)]. Picking appropriate lifts ˜ 1 ri of the (oriented) cells of M to cells of M we get bases σ˜i = {σ ˜i ,..., σ˜i } for the ˜ ޚ[π1(M)]-modules Ci (M), such that if Ai denotes the matrix corresponding to ∂i , then A1 and A3 are of the form t A3 = (1 − g1,..., 1 − gn) , gi ∈ π1(M)

A1 = (1 − h1,..., 1 − hn), hi ∈ π1(M).

Clearly {h1,..., hn} is a generating set for π1(M). Since M is a closed 3-manifold {g1,..., gn} is a generating set for π1(M) as well. In particular we can ﬁnd k, l ∈ {1,..., n} such that φ(gk) 6= 0, φ(hl ) 6= 0. 0 ±1 d In the following we write αi = ϕi ⊗ α : π1(M) → GL(ދ(Gi )[t ] ⊗ ކ ) → 0 [ ±1] = = 1 GL(ދ(Gi ) t , d), i 1, 2 and we write ϕ ϕ2 . Lemma 6.1. We have

deg(α1(1 − hl )) = deg(α2(1 − hl )) = d|φ(hl )|,

deg(α1(1 − gk)) = deg(α2(1 − gk)) = d|φ(gk)|. 0 In particular, the matrices αi (1 − hl ), αi (1 − gk) are invertible over ދ(Gi )(t) for i = 1, 2.

Proof. Note that αi (1 − hl ) = id − αi (hl ), αi (1 − gk) = id − αi (gk) and that φ(hl ) 6= 0, φ(gk) 6= 0. The lemma now follows from Lemma 4.2 since α1 and α2 are φ-compatible.

Denote by B the result of deleting the k-column and the l-row of A2.

Lemma 6.2. τ(M, αi ) 6= 0 if and only if αi (B) is invertible. Furthermore if τ(M, αi ) 6= 0, then −1 −1 0 τ(M, αi ) = αi (1 − gk) αi (B)αi (1 − hl ) ∈ K1(ދ(Gi )(t))/ ± αi (π1(M)). 0 d Proof. Denote the standard basis of ދ(Gi )(t) by e1,..., ed . We equip C j = αi 0 d C j (M; ދ(Gi )(t) ) with the ordered bases

1 1 r j r j Ꮿ j = {σ ˜ j ⊗ e1,..., σ˜ j ⊗ ed ,..., σ˜i ⊗ e1,..., σ˜i ⊗ ed }. Now let ξ0 = ∅, ξ1 = {ld+1,..., l(d+1)},

ξ2 = {1,..., nd}\{kd+1,...,(k+1)d},

ξ3 = {1,..., d}. REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 289

Then ξ = (ξ0, ξ1, ξ2, ξ3) is a τ-chain for C∗. We have A1(ξ) = αi (1−hl ), A2(ξ) = αi (B) and A3(ξ) = αi (1 − gk). Clearly A1(ξ) and A3(ξ) are invertible by Lemma 6.1. The proposition now follows immediately from Theorem 2.1. 6= 0 Now assume that τ(M, α2) 0. Then α2(B) is invertible over ދ(G2)(t) by 0 ±1 0 Lemma 6.2. Note that αi (B) is defined over ޚ[Gi ][t ] ⊂ ދ(Gi )(t). In particular α2(B) = ϕ(α1(B)). It follows from the following lemma that α1(B) is invertible as well. × [ 0 ][ ±1] Lemma 6.3. Let P be an r s matrix over ޚ G1 t . If s r ޚ[G2] → ޚ[G2] v 7→ ϕ(P)v 0 0 is invertible over ދ(G2)(t), then P is invertible over ދ(G1)(t). The same holds with “invertible” replaced by “injective”. 0 Proof. Assume that multiplication by ϕ(P) is injective over ދ(G2)(t). Since [ ] → 0 = : [ ]s → [ ]r ޚ G2 ދ(G2)(t) ދ(G2) is injective it follows that ϕ(P) ޚ G2 ޚ G2 s r is injective. By Proposition 6.4 the map P : ޚ[G1] → ޚ[G1] is injective as 0 = [ ] : 0 s → well. Since ދ(G1)(t) ދ(G1) is flat over ޚ G1 it follows that P ދ(G1)(t) 0 r ދ(G1)(t) is injective. 0 = If ϕ(P) is invertible over the skew field ދ(G2)(t), then r s. But an injective homomorphism between vector spaces of the same dimension over a skew field is 0 in fact an isomorphism. This shows that P is invertible over ދ(G1)(t). s r Proposition 6.4. If G1 is locally indicable, and if ޚ[G1] → ޚ[G1] is a map such [ ]s ⊗ [ ] → [ ]r ⊗ [ ] [ ]s → that ޚ G1 ޚ[G1] ޚ G2 ޚ G1 ޚ[G1] ޚ G2 is injective, then ޚ G1 r ޚ[G1] is injective as well.

Proof. Let K = Ker{ϕ : G1 → G2}. Clearly K is again locally indicable. Note that s r ޚ[G1] → ޚ[G1] can also be viewed as a map between free ޚ[K ]-modules. Pick any right inverse λ : G2 → G1 of ϕ. It is easy to see that g ⊗ h 7→ gλ(h) ⊗ 1, g ∈ G1, h ∈ G2 induces an isomorphism [ ] ⊗ [ ] → [ ] ⊗ ޚ G1 ޚ[G1] ޚ G2 ޚ G1 ޚ[K ] ޚ. s r By assumption ޚ[G1] ⊗ޚ[K ] ޚ → ޚ[G2] ⊗ޚ[K ] ޚ is injective. Since K is locally indicable it follows immediately from [Gersten 1983] or [Howie and Schneebeli s r 1983] (see also [Strebel 1974] for the case of PTFA groups) that ޚ[G1] → ޚ[G1] is injective.

By Lemma 6.2 we have now showed that if τ(M, α2) 6= 0, then τ(M, α1) 6= 0. Furthermore

deg τ(M, αi ) = deg αi (B) − deg αi (1−gk) − deg αi (1−hl ), i = 1, 2. 290 STEFAN FRIEDL

Theorem 1.3 now follows immediately from Lemma 6.1 and from the following proposition. × [ 0 ][ ±1] Proposition 6.5. Let P be an r r matrix over ޚ G1 t . If ϕ(P) is invertible, deg P ≥ deg ϕ(P).

Remark. (1) If ϕ : R → S is a homomorphism of commutative rings, and if P is a matrix over R[t±1], then clearly

deg P = deg det P ≥ deg ϕ(det P) = deg det ϕ(P) = deg ϕ(P).

Similarly, several other results in this paper, e.g. Theorem 3.1 and Lemma 6.1 are clear in the commutative world, but require more effort in our noncommutative setting. [ 0 ] { ∈ [ 0 ]| 6= ∈ [ 0 ]} (2) If (ޚ G1 , f ޚ G1 ϕ( f ) 0 ޚ G2 ) has the Ore property, one can give 0 an elementary proof of the proposition by ﬁrst diagonalizing over ދ(G2) and then 0 over ދ(G1). Since this is not known to be the case, we have to give a more indirect proof. The following proof is based on arguments in [Cochran 2004] and [Harvey 2006]. = : [ 0 ]s → [ 0 ][ ±1]r Proof of Proposition 6.5. Let s deg ϕ(P). Pick a map f ޚ G1 ޚ G1 t such that the induced map

0 s 0 ±1 r 0 ±1 r 0 ±1 r ދ(G2) → ދ(G2)[t ] → ދ(G2)[t ] /ϕ(P)ދ(G2)[t ]

P is an isomorphism. Denote by 0 → C1 −→ C0 → 0 the complex

0 ±1 r P 0 ±1 r 0 → ޚ[G1][t ] −→ ޚ[G1][t ] → 0, → → = [ 0 ]s and denote by 0 D0 0 the complex with D0 ޚ G1 . We have a chain map f D∗ → C∗ given by f : D0 → C0. Denote by Cyl(D∗ −→ C∗) the mapping cylinder of the complexes. We then get a short exact sequence of complexes

f f 0 → D∗ → Cyl(D∗ −→ C∗) → Cyl(D∗ −→ C∗)/D∗ → 0.

More explicitly, we get the commutative diagram

(id id) 0 / C1 ⊕ D0 / C1 ⊕ D0 / 0 P −f (P −f ) 0 id (0 ) 0 / D0 / C0 ⊕ D0 / C0 / 0. REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 291

f Recall that Cyl(D∗ −→ C∗) and C∗ are chain homotopic. Using the deﬁnition of f we therefore see that 0 f 0 f : H0(D∗; ދ(G2)) → H0(Cyl(D∗ −→ C∗), ދ(G2)) 0 is an isomorphism. Since P is invertible over ދ(G2)(t) it follows that

f 0 H1(Cyl(D∗ −→ C∗); ދ(G2)) = 0. It follows from the long exact homology sequence corresponding to the short exact sequence of chain complexes above that

f 0 H1(Cyl(D∗ −→ C∗)/D∗; ދ(G2)) = 0; − 0 thus the matrix P f is injective over ދ(G2). It follows from Lemma 6.3 that

f 0 H1(Cyl(D∗ −→ C∗)/D∗; ދ(G1)) = 0 as well. Again looking at the long exact homology sequence we get that

0 f 0 0 f : H0(D∗; ދ(G1)) → H0(Cyl(D∗ −→ C∗); ދ(G1)) = H0(C∗; ދ(G1)) is an injection. Hence 0 deg ϕ(P) = s = dim 0 (H (D∗; ދ(G ))) ދ(G2) 0 2 0 = dim 0 (H (D∗; ދ(G ))) ދ(G1) 0 1 0 ≤ dim 0 (H (C∗; ދ(G ))) = deg P. ދ(G1) 0 1 6.2. Proof of Theorem 1.3 for 3-manifolds with boundary and for 2-complexes. First let X be a ﬁnite connected 2-complex with χ(X) = 0. We can give X a CW structure with one 0-cell. If n denotes the number n of 1-cells, then n − 1 equals the number of 2-cells. Now consider the chain complex of the universal cover X˜ :

˜ n−1 ∂2 ˜ n ∂1 ˜ 1 0 → C2(X) −→ C1(X) −→ C0(X) → 0. As in Section 6.1 we pick lifts of the cells of X to cells of X˜ to get bases such that if Ai denotes the matrix corresponding to ∂i , then

A1 = (1 − h1,..., 1 − hn).

Note that {h1,..., hn} is a generating set for π1(X). Let l ∈ {1,..., n} such that φ(l) 6= 0. The proof of Lemma 6.2 can easily be modiﬁed to prove the following.

Lemma 6.6. Denote by B the result of deleting the l-row of A2. Then τ(X, α) 6= 0 if and only if α(B) is invertible. Furthermore if τ(X, α) 6= 0, then −1 0 τ(X, α) = α(B)α(1 − hl ) ∈ K1(ދ(Gi )(t))/ ± αi (π1(X)). 292 STEFAN FRIEDL

The proof of Theorem 1.3 for closed manifolds can now easily be modiﬁed to cover the case of 2-complexes X with χ(X) = 0. Now let M be again a 3-manifold whose boundary consists of a nonempty set of = 1 = tori. A duality argument shows that χ(M) 2 χ(∂(M)) 0. Clearly M is homotopy equivalent to a 2-complex. Reidemeister torsion is not a homotopy invariant but the following lemma still allows us to reduce the case of a 3-manifold with boundary to the case of a 2-complex.

Lemma 6.7 [Turaev 2001, p. 56 and Theorem 9.1]. Let M be a 3-manifold with boundary. Then there exists a 2-complex X and a simple homotopy equivalence ∼ M → X. In particular, if α : π1(X) = π1(M) → GL(R, d) is a representation such d that H∗(X, R ) = 0, then

τ(M, α) = τ(X, α) ∈ K1(R)/ ± α(π1(M)).

Theorem 1.3 for 3-manifolds with boundary now follows from Theorem 1.3 for 2-complexes X with χ(X) = 0.

7. Harvey’s monotonicity theorem for groups

Let π be a finitely presented group and let (ϕ : π → G, φ : π → ޚ) be an admissible 0 0 pair for π. Consider G = G (φG) and pick a splitting ޚ → G of φG. As in Section 5.2 we can consider the skew Laurent polynomial ring ދ(G0)[t±1] together with the φ-compatible representation π → GL(ދ(G0)[t±1], 1). Following [Harvey 2006, Definition 1.6] we define δG(φ) to be zero if the group 0 ±1 0 ±1 H1(π, ދ(G )[t ]) is not ދ(G )[t ]-torsion and

0 ±1 δG(φ) = dimދ(G0)(H1(π, ދ(G )[t ])) otherwise. We give an alternative proof for the following result of Harvey [2006, Theorem 2.9].

Theorem 7.1. Let π = π1(M), where M is a closed 3-manifold. If (ϕ1 : π → G1, ϕ2 : π → G2, φ) is an admissible triple for π, then ≥ δG1 (φ) δG2 (φ) if (ϕ2, φ) is not initial, ≥ − δG1 (φ) δG2 (φ) 2 otherwise.

Proof. We clearly only have to consider the case that δG2 (φ) > 0. We can build K (π, 1) by adding i-handles to M with i ≥ 3. It therefore follows that for the admissible pairs (ϕi : π → Gi , φ) we have

0 ±1 0 ±1 δ (φ) = dim 0 (H (K (π, 1); ދ(G )[t ])) = dim 0 (H (M; ދ(G )[t ])). Gi ދ(Gi ) 1 1 i ދ(Gi ) 1 i REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 293

We combine this equality with Theorem 1.3, Corollary 3.6 and Lemmas 3.2, 4.3, 4.4. The theorem follows now immediately from the observation that Im{π1(M) → 0 ±1 Gi → GL(ދ(Gi )[t ], 1)} is cyclic if and only if φ : Gi → ޚ is an isomorphism. This monotonicity result gives in particular an obstruction for a group π to be the fundamental group of a closed 3-manifold. For example, Harvey [2006, Example 3.2] shows that as an immediate consequence we get the well-known fact that ޚm, m ≥ 4, is not a 3-manifold group.

Remark. In [Friedl and Kim 2005] we considered the case π = π1(M), where M is a closed 3-manifold. Given an admissible pair (ϕ : π → G, φ) we show (under a mild assumption) that δG(φ) is even, generalizing [Turaev 1986, p. 141]. In [Friedl and Harvey 2006] it is shown that given π → G, G locally indicable and amenable, the map Hom(G, ޚ) → ޚ φ 7→ δG(φ) defines a seminorm on Hom(G, ޚ). Let π be a finitely presented group of deficiency at least one, for example π = π1(M) where M is a 3-manifold with boundary. Using a presentation of deficiency one we can build a 2-complex X with χ(X) = 0 and π1(X) = π. The same proof as the proof of Theorem 7.1 now gives the following theorem of Harvey. (In the 3 case that π = π1(S \ K ) for K a knot, this was first proved in [Cochran 2004].) Theorem 7.2 [Harvey 2006, Theorem 2.2]. If π is a finitely presented group of deficiency one and if (ϕ1, ϕ2, φ) is an admissible triple for π, then ≥ δG1 (φ) δG2 (φ) if (ϕ2, φ) is not initial, ≥ − δG1 (φ) δG2 (φ) 1 otherwise.

8. Open questions and problems

Let M be a 3-manifold and φ ∈ H 1(M; ޚ). We propose the following three problems for further study.

(1) If (ϕ : π1(M) → G, φ) is an admissible pair for π1(M) and if α : π1(M) → GL(ކ, d) factors through ϕ, does it follow that 1 deg τ(M, α) ≤ deg τ(M, ޚ[π (M)] → ދ(G0)(t))? d 1 Put differently, are the Thurston norm bounds of Cochran and Harvey optimal, i.e., at least as good as the Thurston norm bounds of [Friedl and Kim 2006] for any representation factoring through G? 294 STEFAN FRIEDL

0 (2) In many cases deg τ(M, ޚ[π1(M)] → ދ(G )(t)) < kφkT , for any admissible pair (ϕ : π1(M) → G, φ). For example this is the case if K is a knot with 1K (t) = 1 and M = X (K ). It is an interesting question whether invariants can be defined for any map π1(M) → G, G a (locally indicable) torsion-free group. For example it might be possible to work with ᐁ(G) the algebra of affiliated operators (see [Reich 1998], for instance) instead of ދ(G). If such an extension is possible, then it is a natural question whether the Thurston norm is determined by such more general bounds. This might be too optimistic in the general case, but it could be true in the case of a knot complement. (3) If (M, φ) fibers over S1, the corresponding Alexander polynomial defined over ޚ[t±1] is monic, that is, the top coefficient is ±1. Because of the high degree of indeterminacy of Alexander polynomials over skew Laurent polynomial rings a corresponding statement is meaningless. Since Reidemeister torsion has a much smaller indeterminacy it is potentially possible to use it to extend the fiberedness obstruction as in [Goda et al. 2005].

Acknowledgment

The author thanks Stefano Vidussi for pointing out the functoriality of Reidemeister torsion, and Tim Cochran, Shelly Harvey and Taehee Kim for helpful discussions. The author also thanks the referee for many helpful comments.

References

[Chapman 1974] T. A. Chapman, “Topological invariance of Whitehead torsion”, Amer. J. Math. 96 (1974), 488–497. MR 52 #11931 Zbl 0358.57004 [Cochran 2004] T. D. Cochran, “Noncommutative knot theory”, Algebr. Geom. Topol. 4 (2004), 347–398. MR 2005k:57023 Zbl 1063.57011 [Cochran et al. 2003] T. D. Cochran, K. E. Orr, and P. Teichner, “Knot concordance, Whitney towers and L2-signatures”, Ann. of Math. (2) 157:2 (2003), 433–519. MR 2004i:57003 Zbl 1044.57001 [Cohn 1985] P. M. Cohn, Free rings and their relations, 2nd ed., London Mathematical Society Monographs 19, Academic Press, London, 1985. MR 87e:16006 Zbl 0659.16001 [Dodziuk et al. 2003] J. Dodziuk, P. Linnell, V. Mathai, T. Schick, and S. Yates, “Approximating L2- invariants and the Atiyah conjecture”, Comm. Pure Appl. Math. 56:7 (2003), 839–873. Dedicated to the memory of Jürgen K. Moser. MR 2004g:58040 Zbl 1036.58017 [Eisermann 2000] M. Eisermann, “The number of knot group representations is not a Vassiliev invariant”, Proc. Amer. Math. Soc. 128:5 (2000), 1555–1561. MR 2000j:57009 Zbl 0939.57007 [Friedl and Harvey 2006] S. Friedl and S. Harvey, “Non-commutative multivariable Reidemeister torsion and the Thurston norm”, preprint, 2006. math.GT/0608409 [Friedl and Kim 2005] S. Friedl and T. Kim, “The parity of the Cochran–Harvey invariants of 3- manifolds”, preprint, 2005. To appear in Trans. Amer. Math. Soc. math.GT/0510475 [Friedl and Kim 2006] S. Friedl and T. Kim, “The Thurston norm, ﬁbered manifolds and twisted Alexander polynomials”, Topology 45:6 (2006), 929–953. MR MR2263219 Zbl 1105.57009 REIDEMEISTER TORSION, THE THURSTON NORM AND HARVEY’S INVARIANTS 295

[Gersten 1983] S. M. Gersten, “Conservative groups, indicability, and a conjecture of Howie”, J. Pure Appl. Algebra 29:1 (1983), 59–74. MR 84m:20035 Zbl 0513.20017 [Goda et al. 2005] H. Goda, T. Kitano, and T. Morifuji, “Reidemeister torsion, twisted Alexander polynomial and fibered knots”, Comment. Math. Helv. 80:1 (2005), 51–61. MR 2005m:57008 Zbl 1066.57008 [Harvey 2005] S. L. Harvey, “Higher-order polynomial invariants of three-manifolds giving lower bounds for the Thurston norm”, Topology 44:5 (2005), 895–945. MR 2006g:57019 Zbl 1080.57019 [Harvey 2006] S. L. Harvey, “Monotonicity of degrees of generalized Alexander polynomials of groups and 3-manifolds”, Math. Proc. Cambridge Philos. Soc. 140:3 (2006), 431–450. MR 2007c: 57004 Zbl 05025112 [Higman 1940] G. Higman, “The units of group-rings”, Proc. London Math. Soc. (2) 46 (1940), 231–248. MR 2,5b JFM 0025.24302 [Howie and Schneebeli 1983] J. Howie and H. R. Schneebeli, “Homological and topological properties of locally indicable groups”, Manuscripta Math. 44:1-3 (1983), 71–93. MR 85c:20041 Zbl 0533.20022 [Kirk and Livingston 1999] P. Kirk and C. Livingston, “Twisted Alexander invariants, Reidemeis- ter torsion, and Casson-Gordon invariants”, Topology 38:3 (1999), 635–661. MR 2000c:57010 Zbl 0928.57005 [Leidy and Maxim 2006] C. Leidy and L. Maxim, “Higher-order Alexander invariants of plane algebraic curves”, Int. Math. Res. Not. 2006 (2006), ID 12976. MR MR2264729 Zbl 05122980 [Lickorish 1997] W. B. R. Lickorish, An introduction to knot theory, Graduate Texts in Mathematics 175, Springer, New York, 1997. MR 98f:57015 Zbl 0886.57001 [Lin 2001] X. S. Lin, “Representations of knot groups and twisted Alexander polynomials”, Acta Math. Sin. (Engl. Ser.) 17:3 (2001), 361–380. MR 2003f:57018 Zbl 0986.57003 [McMullen 2002] C. T. McMullen, “The Alexander polynomial of a 3-manifold and the Thurston norm on cohomology”, Ann. Sci. École Norm. Sup. (4) 35:2 (2002), 153–171. MR 2003d:57044 Zbl 1009.57021 [Milnor 1966] J. Milnor, “Whitehead torsion”, Bull. Amer. Math. Soc. 72 (1966), 358–426. MR 33 #4922 Zbl 0147.23104 [Ranicki 1998] A. Ranicki, High-dimensional knot theory: Algebraic surgery in codimension 2, Springer, New York, 1998. MR 2000i:57044 Zbl 0910.57001 [Reich 1998] H. Reich, Group von Neumann algebras and related algebras, Thesis, University of Göttingen, 1998. [Rosenberg 1994] J. Rosenberg, Algebraic K -theory and its applications, Graduate Texts in Mathe- matics 147, Springer, New York, 1994. MR 95e:19001 Zbl 0801.19001 [Stenström 1975] B. Stenström, Rings of quotients: an introduction to methods of ring theory, Grundlehren der Mathematischen Wissenschaften 217, Springer, New York, 1975. MR 52 #10782 Zbl 0296.16001 [Strebel 1974] R. Strebel, “Homological methods applied to the derived series of groups”, Comment. Math. Helv. 49 (1974), 302–332. MR 50 #7373 Zbl 0288.20066 [Tamari 1957] D. Tamari, “A refined classification of semi-groups leading to generalized polynomial rings with a generalized degree concept”, pp. 439–440 in Proceedings of the International Congress of Mathematicians (Amsterdam, 1954), vol. 3, E. P. Noordhoff, Groningen, 1957. [Thurston 1986] W. P. Thurston, A norm for the homology of 3-manifolds, Mem. Amer. Math. Soc. 339, Amer. Math. Soc., Providence, RI, 1986. MR 88h:57014 Zbl 0585.57006 296 STEFAN FRIEDL

[Turaev 1986] V. G. Turaev, “Reidemeister torsion in knot theory”, Uspekhi Mat. Nauk 41:1 (1986), 97–147. In Russian; translated in Russian Math. Surveys 41;1 (1986), 119–182. MR 87i:57009 Zbl 0602.57005 [Turaev 2001] V. Turaev, Introduction to combinatorial torsions, Lectures in Mathematics, ETH Zürich, Birkhäuser, Basel, 2001. MR 2001m:57042 Zbl 0970.57001 [Turaev 2002a] V. Turaev, “A homological estimate for the Thurston norm”, preprint, Univ. Louis Pasteur, Strasbourg, 2002. math.GT/0207267 [Turaev 2002b] V. Turaev, “A norm for the cohomology of 2-complexes”, Algebr. Geom. Topol. 2 (2002), 137–155. MR 2002m:57005 Zbl 1014.57002

Received August 31, 2005. Revised December 13, 2005.

STEFAN FRIEDL DEPARTEMENT´ DE MATHEMATIQUES´ UQAM C.P. 8888, SUCCURSALE CENTRE-VILLE MONTREAL´ ,QC H3C 3P8 CANADA [email protected] http://www.labmath.uqam.ca/~friedl/index.html PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

ARF INVARIANTS OF REAL ALGEBRAIC CURVES

PATRICK M.GILMER

Dedicated to the memory of my adviser, P. Emery Thomas

We give new congruences for singular real algebraic curves, generalizing Fiedler’s congruence for nonsingular curves.

1. Introduction

Let Aޒ be an irreducible real algebraic curve of degree 2k with only real nodal singularities. Aޒ consists of the image of a number of immersed circles. Aޒ is called an M-curve if the number of immersed circles plus the number of double + 2k−1 points is 1 2 . Let Aރ be the complex curve in ރސ(2) given by the same polynomial as Aޒ. Thus Aޒ = Aރ ∩ ޒސ(2). Aޒ is an M-curve precisely when Aރ \ Aޒ consists of two punctured spheres interchanged by complex conjugation. + Arbitrarily choose one of these components, say Aރ . The complex structure on + + Aރ induces an orientation on Aރ , and thus on each immersed circle of Aޒ. Of course if we choose the other component, we get the opposite orientation on each immersed circle of Aޒ. An orientation on each of the components up to reversing all the orientations simultaneously is called a semiorientation. Thus each M-curve receives a semiorientation, called the complex orientation [Rokhlin 1978]. An oval is a two-sided simple closed curve in the real projective plane ޒސ(2). The inside of an oval is the component of its complement that is a disk; the outside is a Mobius¨ band. Suppose Ꮿ is a simple curve, that is, a disjoint collection of oriented ovals. An oval of Ꮿ is called even or odd according to whether it lies inside an even or odd number of other ovals of Ꮿ. Let p(Ꮿ) denote the number of even ovals in Ꮿ, and n(Ꮿ) the number of odd ovals in Ꮿ. If one of ovals of Ꮿ lies inside another oval of Ꮿ or vice versa, we say they are linked. We say Ꮿ is odd if each oval is linked with an odd number of other ovals; thus an odd curve must have an even number of components. We say Ꮿ is even if each oval is linked with an even number of other ovals and the total number of ovals is odd.

MSC2000: primary 14P25; secondary 57M27. Keywords: nodal curve, oval, link.

297 298 PATRICK M. GILMER

We denote by 5+(Ꮿ) the number of pairs of linked ovals for which the orientations on the curves extend to an orientation of the intervening annulus, and by 5−(Ꮿ) the number of pairs of linked ovals for which this is not the case. We will usually write simply n, p, 5± without Ꮿ, except in ambiguous instances. By a curve we will mean a collection of immersed oriented curves in ޒސ(2) with only transverse double point intersections. We say two curves A and A0 are weakly equivalent if they can be connected via a sequence of ambient isotopies in ޒސ(2), plus local moves and their inverses: balanced type I moves (see ﬁgure), safe type II moves, type III moves, and “empty” ﬁgure-eight deaths.1

A balanced type I move replaces an arc having two curls, as shown, by an arc without double points.

Theorem 1.1. Let Aޒ be a nodal M-curve of degree 2k. If k is even and Aޒ is weakly equivalent to an odd simple curve Ꮿ, then + − − − ≡ 1 2 1 2 − 5 (Ꮿ) 5 (Ꮿ) p(Ꮿ) 2 k or ( 2 k 2)(mod 8).

If k is odd and Aޒ is weakly equivalent to an even simple curve Ꮿ, then + − − − ≡ 1 2 − 5 (Ꮿ) 5 (Ꮿ) n(Ꮿ) 2 (k 1)(mod 8).

In the case Aޒ = Ꮿ, this reduces to a congruence due to Fiedler [1983]. The reduction is not obvious; we prove it explicitly in Theorem 5.4. Fiedler [1986] has also given generalizations of his congruence to singular curves. The scheme

for a degree-8 nodal M-curve with complex orientation is prohibited by Theorem 1.1 but not by the results of [Fiedler 1986]. Nor is it prohibited by any of the

1Type II and III moves mean Reidemeister moves of these types on diagrams without over- or undercrossings. In the type II case, “safe” means we require in addition that the two strands have opposite orientations. ARFINVARIANTSOFREALALGEBRAICCURVES 299 other known general restrictions on nodal curves: the Kharlamov–Viro congruences [1988] (as correctly stated in [Viro and Orevkov 2001]), the Viro inequalities [Viro 1978; Finashin 1996; Gilmer 2000], or the extremal properties of the Viro inequalities in [Gilmer 2000]. If we perturb the figure-eights in this scheme into pairs of ovals we obtain the scheme for a nonsingular M-curve h14 ` 1h7ii which can be realized by a real algebraic curve [Viro 1986]. Theorem 1.1 is a corollary of Theorem 5.3 below, which in turn is a simplified version of the more general Theorem 3.3 of [Gilmer 2000]. To prove (or even state) Theorem 5.3, we must discuss the Arf invariant of links, which we do in Sections 2 and 3. These sections are, mainly, a review of parts of [Gilmer 1993a; 1993b]. In Section 6 we show that the two explicit examples given by Fiedler of curves prohibited by his congruence [1986] for singular curves are prohibited by our Theorem 5.3 as well. It seems likely that any scheme for a curve with complex orientation that can be prohibited by Fiedler’s congruence can also be prohibited by Theorem 5.3. However our theorem applies to hypothetical curves with a complex orientation. Fiedler’s result concerns hypothetical singular curves which, by hypothesis, are related to actual nonsingular real algebraic curves by desingulariza- tion. Thus information about the complex orientations of the hypothetical curves is contained only implicitly in the relation to the actual nonsingular real algebraic curve. For this reason, it seems difficult to derive [Fiedler 1986] as a corollary of Theorem 5.3 in a way similar to the proof of Theorem 5.4. However both obstruc- tions can be interpreted as deriving from the calculation of the Brown invariant of the Gillou–Marin form on characteristic surfaces. Moreover the two surfaces are closely related.

2. Arf invariants of links in S3

An oriented link in S3 is called proper if for each component the sum of the linking numbers with all the other components is even. That is, L = ti Ki is proper if and only if lk(Ki , L − Ki ) ≡ 0 (mod 2) for all i. We use lk to denote the ޚ-valued linking number of oriented links. Robertello defined the Arf invariants of proper links and gave several equivalent definitions. One involved the Seifert pairing on an orientable spanning surface. We generalized this definition so that it applies to nonorientable spanning surfaces [Gilmer 1992; 1993a]. This definition for the Arf invariant is analogous to the Gordon–Litherland [Gordon and Litherland 1978] definition of the signature of a knot. There is a version for unoriented links, but the oriented link version is more useful in this paper. Let V be a ޚ/2ޚ vector space equipped with a symmetric bilinear form

·: V × V → ޚ/2ޚ. 300 PATRICK M. GILMER

A function q : V → ޚ/4ޚ is called a quadratic reﬁnement of · if

q(x+y) − q(x) − q(y) = 2 x · y for all x, y ∈ V . Here 2 denotes the nontrivial group homomorphism ޚ/2ޚ → ޚ/4ޚ. Let rad be the radical of · . We say that q is proper if q vanishes on rad. If q is proper, the Brown invariant β(q) ∈ ޚ/8ޚ is defined by the equation 1 X e2πiβ(q)/8 = √ iq(v). dim V +dim rad ( 2) v∈V The Brown invariant is additive for the direct sum of quadratic refinements. A simple graphical scheme for writing a given form as a direct sum of simple elementary forms and thus calculating the Brown invariant is given in [Gilmer 1993a; 1993b]. 3 Let L be a link in S with oriented components {Ki }. Let F be a not necessarily orientable spanning surface for L. Define a map

qF : H1(F, ޚ/2ޚ) → ޚ/4ޚ as follows. Given x ∈ H1(F, ޚ/2ޚ), pick a simple closed curve αx representing x, and deﬁne qF (x) to be the number of positive half twists in a tubular neighborhood of x in F. More precisely,

qF (x) = lk(αx , αˆ x ).

Here αˆ x is the boundary of a tubular neighborhood of αx oriented in the same direction as some arbitrarily chosen orientation for x. The function qF is well deﬁned and is a quadratic reﬁnement of the intersection pairing

·: H1(F, ޚ/2ޚ) × H1(F, ޚ/2ޚ) → ޚ/2ޚ. We note that

qF ([Ki ]) = 0 ⇐⇒ lk(Ki , L − Ki ) ≡ 0 (mod 2). Let F denote the quotient space of F obtained by identifying each component of F to a point corresponding to that component. Let π : F → F denote the quotient map. Thus L is proper if and only if qF [Ki ] = 0 for all i if and only if qF can be factored through π∗. Deﬁne = 1 X 0 µ(F) 2 lk(Ki , K j ), i, j 0 where K j denotes K j pushed slightly into the interior of F. Then set

Arf L = β(qF ) − µ(F) ∈ ޚ/8ޚ. ARFINVARIANTSOFREALALGEBRAICCURVES 301

One can relate any two spanning surfaces by a sequence of moves of certain types (and their inverses): (0) isotopy; (1) adding a hollow handle; (2) replacing a collar of the boundary with a punctured Mobius¨ band, obtained by adding an unknotted positively or negatively half- twisted band (see Figure 7 in [Gordon and Litherland 1978]); (3) the birth of an empty two-sphere in a small 3-ball, as described in [Gilmer 1993a]. The equivalence relation generated by these moves is called S∗-equivalence. Since β(qF ) − µ(F) is preserved by these moves, Arf L is well deﬁned. It takes values in 4ޚ/8ޚ. It is invariant under oriented band summing, and adding and removing small unlinked unknots. This implies that it is an invariant of planar cobordism [Gilmer 1993a]. Consider the trefoil below, which is spanned by a Mobius¨ band F. The group H1(F, ޚ/2ޚ)≈ޚ/2ޚ is generated by the core x. We have qF (x)≡−3≡1 (mod 4), ≡ = −6 = − ≡ β(qF ) 1 (mod 8), µ 2 3, and Arf(trefoil) 4 (mod 8).

3. Arf invariants of links in rational homology spheres

Let L be a link in a rational homology sphere M with oriented components {Ki }. By a spanning surface for L we mean a possibly nonorientable surface F in M with boundary L. We can speak of S∗-equivalence of spanning surfaces in M. Two spanning surfaces for a link need not be S∗-equivalent. We introduced a parameter to index S∗ equivalence classes of spanning surfaces in [Gilmer 1993a]. Suppose the homology class of L represents zero in H1(M, ޚ/2ޚ). Then

0(L) = {γ ∈ H1(M)|2γ = [L] ∈ H1(M)} is nonempty. Let F be a spanning surface for L with no closed components, and iF : F→M be the inclusion. H1(F) is free abelian and the homology class of L the boundary 302 PATRICK M. GILMER of F equipped with the string orientation of the link represents a homology class in H1(F) which is divisible by two. Define = 1 [ ] ∈ ∈ ⊂ γ (F) iF ( 2 L H1(F)) 0(L) H1(M). If F has some closed components, we take γ (F) to be γ (F0), where F0 is formed by deleting the interior of a disk from each closed component and orienting the new boundary components in an arbitrary way. For further discussion, see [Gilmer 1992]. Then γ (F) is preserved by S∗-equivalence, and the map from S∗-equivalence classes of spanning surfaces for L to 0(L) is bijective. By abuse of notation, we let Fγ denote a spanning surface F with γ (Fγ ) = γ . This should not cause any confusion. The number of positive half twists in a tubular neighborhood of a curve on a surface in M has no well-defined analog in this more general situation. What we actually need is an analog of the number of half twists modulo four. We do have a linking number in ޑ which can be defined for disjoint 1-cycles. We continue to denote this linking number by lk. We do know what it means to increase or decrease the number of half twists in the neighborhood of a curve α. Moreover if we add a half twist to a neighborhood of α, we increase the linking number of α and the boundary of the neighborhood of α by one. But we do not know what untwisted (mod 4) should mean. As a replacement for this, it suffices to fix a quadratic refinement of the linking form of M [Gilmer 1993b]. The linking form of M,

`M : H1(M) × H1(M)→ޑ/ޚ, is a bilinear form with an injective adjoint. A quadratic reﬁnement of `M is a function r : H1(M)→ޑ/ޚ, such that r(x + y) − r(x) − r(y) ≡ `M (x, y)(mod 1). It follows that the boundary αˆ of a neighborhood of an oriented curve α on a surface F will have lk(α,ˆ α) ≡ r(3[α]) −r(2[α]) −r([α]) ≡ (33 − 22 − 1)r([α]) ≡ 4r([α])(mod 1). Thus we may deﬁne

qr,F : H1(F, ޚ/2ޚ) → ޚ/4ޚ by qr,F (x) ≡ lk(γx , γˆx ) − 4r(x)(mod 4).

This is a quadratic reﬁnement of the intersection form on H1(F, ޚ/2ޚ), and is well deﬁned by [Gilmer 1993a, Theorem 6.1]. ARFINVARIANTSOFREALALGEBRAICCURVES 303

We also have, by [Gilmer 1993a, Proposition 6.3], qr,F ([Ki ]) = 0 ⇐⇒ lk(Ki , L − Ki ) ≡ 2`M ([Ki ], γ (F)) − 2r([Ki ])(mod 2). [ ] = Note that qr,Fγ Ki 0 for all i if and only if qr,Fγ can be factored through π∗. We say (L, γ, r) is proper if either of these equivalent conditions holds, that is, if

lk(Ki , L − Ki ) ≡ 2`M ([Ki ], γ (F)) − 2r([Ki ])(mod 2) for all i. As in the previous section, we deﬁne

1 X 0 µ(F ) = lk(Ki , K ) γ 2 j i, j 0 where K j denotes K j pushed slightly into the interior of Fγ . If (L, γ, r) is proper, we define = − ∈ (3-1) Arf(L, γ, r) β(qr,Fγ ) µ(Fγ ) ޑ/8ޚ. As for proper links in S3, Arf is well defined and is an invariant of planar cobordism. It is also invariant under oriented band summing, and adding and removing small unlinked unknots. When defined, Arf(L, γ, r), taken modulo four, depends only on γ and r [Gilmer 1993a, Proposition 6.9]. (In [Gilmer 1993a; 1996] our µ above is broken up into the sum of two terms, = P − 1 = 1 P 0 λ(L) i< j lk(Ki , K j ) and 2 e(Fγ ) 2 i lk(Ki , Ki ). For our purposes in this paper, the use of µ makes things simpler.) We note that

µ(Fγ ) ≡ `([L], γ ) (mod 1). This restricts the range of values Arf(L, γ, r) can take. 1 Both 0(L) and the set of all quadratic reﬁnements of `(M) are free H (M, ޚ2) sets. Properness of links is preserved under the action which changes both γ and r simultaneously by [Gilmer 1996, 6.5]. Moreover Arf(L, γ, r) changes in a nice way under this action Proposition 3.1 [Gilmer 1996, 6.5]. If (L, γ, q) is proper then (L, ψ · γ, ψ · q) is proper and

Arf(L, ψ · γ, ψ · q) = Arf(L, γ, q) + Arf(∅, ψ · 0, ψ · q).

4. The tangent circle bundle of the real projective plane

We will be concerned with links in the tangent circle bundle of ޒސ(2) which we will denote ᐀. This is the boundary of Ᏸ, the tangent disk bundle of ޒސ(2). Ᏸ, of course, has the homology of ޒސ(2). It follows that ᐀ is a rational homology sphere. In fact ᐀ is the lens space L(4, 3) [Gilmer 1992]. We let ` denote `᐀. 304 PATRICK M. GILMER

By an Ᏽ-curve, we mean a collections of immersed oriented curves in ޒސ(2) where each curve is never tangent to itself or any other curve in the collection. We can describe links in ᐀ by Ᏽ-curves. Lying over an Ᏽ-curve C in ᐀, we have a link ᐀(C) consisting of all the rays tangent to a part of C and pointing in the direction of the orientation. We let g denote the homology class of ᐀ represented by the oriented knot in ᐀ which lies over a straight line with either orientation. (There is an isotopy that can be obtained by spinning the line around a point on the line. It sends a line in ޒސ(2) with one orientation to the same line with the opposite orientation.) The homology class g is a generator for H1(᐀). An oval represents 2g. Whenever we have an invariant of links in ᐀, we obtain an invariant of Ᏽ-curves. We use the same symbol to denote an invariant of oriented links in ᐀ and the corresponding invariant of Ᏽ-curves. An immersed circle in an Ᏽ-curve C is called a component of C, and describes a component of the link ᐀(C). This should not cause any confusion and simplifies our expressions. Similarly, if we have two Ᏽ-curves C1 and C2 which are never tangent to each ∈ 1 other, we can speak of their linking number lk(C1, C2) 4 ޚ. In [Gilmer 1992], we worked out the linking numbers of some two component Ᏽ-curves. Two linked ovals (see Section 1) oriented in the same direction (so they contribute to 5−) have linking number 1. Two linked ovals that are oriented oppositely (so they contribute to 5+) have linking number −1. Two unlinked ovals have linking number zero. 1 A one-sided simple closed curve and a disjoint oval have linking number 2 , if the oval is homologous to twice the one-sided curve in the Mobius¨ band formed by deleting the interior of the oval from ޒސ(2). Otherwise the linking number − 1 is 2 . A one-sided simple closed curve and oval which the curve meets twice − 1 transversally have linking number 2 . Two one-sided simple closed curves which − 1 = − 1 meet in one point have linking number 4 . Thus `(g, g) 4 , and there are two = −1 = 3 quadratic refinements of `, r−1/8 with r−1/8(g) 8 , and, r3/8 with r3/8(g) 8 . A dangerous type II move (that is, a nonsafe type II Reidemeister move on diagrams) between two components which reduces the number of double points by two leads to a new Ᏽ-curve where the linking number between the two components has been increased by one. Smoothing (according to the orientations) a double point of an Ᏽ-curve is called a smoothing move. A smoothing move corresponds to a oriented band move to the corresponding link in ᐀. Thus if we wish to calculate a linking number between two sub-Ᏽ-curves of an Ᏽ-curve, we can smooth the double points of sub-Ᏽ-curves without changing the relevant linking numbers. Of course the reverse of this smoothing move which we call an unsmoothing move will introduce a double point and also corresponds to an oriented band move performed on the corresponding link in ᐀. Lying above an empty figure-eight curve is a local unknot in ᐀. The easiest way to see this is to note that any empty figure-eight is isotopic to an Ᏽ-curve ARFINVARIANTSOFREALALGEBRAICCURVES 305 which does not use every tangent direction as one travels around a circuit. Thus this Ᏽ-curve lies in an interval bundle over a disk in ޒސ(2) which includes the figure-eight. Moreover the Ᏽ-curve projects to a curve in this disk with only one double point. Also, lying above two Ᏽ-curves related by balanced type I move, a safe type II or a type III move are isotopic links in ᐀. Thus weakly equivalent Ᏽ-curves are proper for that same γ and r, and have the same Arf invariant, when it is defined. As an example, consider the Ᏽ-curve, called td given by the diagram for trefoil in Figure 2, with the three crossings (made double points) placed in an affine part of ޒސ(2).AMobius¨ band F is described by a “vector field” 2 on the shaded region which extends the tangential field on the boundary. Let δ denote a 1-sided curve in F which represents the homology class 2g. Thus γ (F) = 2g. The linking number of the boundary of F with a parallel may be calculated using the above techniques. It is −2. Thus µ(F) = −1. Moreover it follows that

(4-1) q−1/8,F (δ) = q3/8,F (δ) = 1. −1 ≡ 3 ≡ Thus Arf(td, 2g, 8 ) Arf(td, 2g, 8 ) 2 (mod 8). td is actually a nodal M-curve of degree 4. This is a small conﬁrmation of Theorem 5.3 below. Proposition 4.1.

Arf(∅, 2g, r−1/8) ≡ 2 (mod 8) and Arf(∅, 2g, r3/8) ≡ −2 (mod 8).

If either (L, γ, r−1/8) or (L, γ + 2g, r3/8) is proper for some γ , then the other is proper, and

Arf(L, γ, r− 1 ) − Arf(L, γ + 2g, r3/8) ≡ 2 (mod 8). 8 Proof. Fix a line in ޒސ(2) and consider the Klein bottle K in ᐀ given by the set of all directions through points on this line. K is a spanning surface for the empty link. The lift of this line with one orientation α, and the lift α0 with the other orientation each have neighborhoods which are Mobius¨ bands. These neighborhoods may be isotoped off of K so that they are the two lifts to ᐀ of the Mobius¨ bands ˆ 0 ˆ 0 1 neighborhood of the line in ޒސ(2). Since lk(α, α) and lk(α , α ) are seen to be 2 , we calculate that = 0 = 1 − · −1 = (4-2) qr−1/8,K (α) qr−1/8,K (α ) 2 4 8 1, = 0 = 1 − · 3 = − (4-3) qr3/8,K (α) qr3/8,K (α ) 2 4 8 1. In this way we obtain the ﬁrst two equations. The rest follows from Proposition 3.1. 2At the double points, we have a whole arc of lines joining the two intersecting lines which ﬁll out the shaded region. 306 PATRICK M. GILMER

The following is a special case of [Gilmer 2000, Proposition 3.2]. Its proof is a simple calculation using as a spanning surface the union of k disjoint annuli swept out by rotating k lines.

Proposition 4.2. Let X2k be 2k lines in general position. Then (X2k, kg, r−1/8) and (X2k,(k + 2)g, r3/8) are proper, and ≡ 1 2 ≡ + + Arf(X2k, kg, r−1/8) 2 k Arf(X2k,(k 2)g, r3/8) 2 (mod 8).

Also (X2k,(k + 2)g, r−1/8) and (X2k, kg, r3/8) are not proper. If C is a collection of disjoint ovals in ޒސ(2), let B+(C) be the closed surface in ޒސ(2) with boundary C which is the closure of the set of points which lie inside an odd number of ovals of C. Let B−(C) be the closure of ޒސ(2) \ B+(C). Proposition 4.3. Let C be a collection of disjoint ovals in ޒސ(2) with an even number of components. There is equivalence between: (1) (C, γ, r) is proper for some γ and r. (2) (C, γ, r) is proper for all possible γ (i.e., both 0 and 2g) and for all r. (3) C is odd. (4) Every component of B+(C) has even Euler characteristic.

Proof. Each component Ci represents 2g ∈ H1(᐀). Thus 2r([Ci ]) = 8r[g] = 1 (mod 2). As C has an even number of components and each oval represents 2g, we have that [C] = 0 ∈ H1(᐀). Thus γ must be either 0 or 2g. Also C is proper for either γ and either r if and only if lk(Ci , C \ Ci ) is odd. On the other hand, lk(Ci , C \ Ci ) is odd if and only if C is odd. The equivalence of the last two conditions is easily seen. The deﬁnitions of this paragraph are due to Rokhlin [1978]. A linked pair of ovals is called positive or negative according to whether the pair contributes to 5+ or 5−. An odd oval is called disoriented if forms a negative pair with the even oval that immediately surrounds it. Let d denote the number of disoriented ovals. Let D+ denote the number of positive pairs with disoriented outer oval. Similarly let D− denote the number of negative pairs with disoriented outer oval. Rokhlin observed that 5+ − 5− = n − 2(d − D+ + D−). If C is odd, it is easy to see that D+ + D− is even. Thus, if C is odd, we have

(4-4) 2d ≡ 5+ − 5− − n (mod 4).

Proposition 4.4. Let C be an odd collection of disjoint ovals in ޒސ(2). Then (C, (5+ − 5− − p)g, r) and (C, (5+ − 5− − p − 2)g, r) are proper for either r. ARFINVARIANTSOFREALALGEBRAICCURVES 307

Moreover + − + − Arf(C, (5 − 5 − p)g, r−1/8) ≡ Arf(C,(5 −5 − p)g,r3/8) ≡ 5+ −5− − p (mod 8), + − + − Arf(C, (5 − 5 − p − 2)g, r3/8) ≡ 5 −5 − p−2 (mod 8), + − + − Arf(C,(5 −5 − p+2)g,r−1/8) ≡ 5 −5 − p+2 (mod 8). Proof. By Proposition 4.3, every component of B+ has even Euler characteristic. Thus we can pick a vector field on B+ which is tangent to the boundary and pointed in the direction of the orientation of C with (n− p)/2 zeros of index -2. This defines a surface in ᐀ lying over B+ with (n− p)/2 disks removed around the zeros, which we denote by zi . We complete this surface to form a spanning surface F for L(C) in ᐀ by adding Mobius¨ bands above the disks around the zeros. We do this so that the cores of these Mobius¨ bands, which we denote by xi , are the fibers over zi of the map from F to ޒސ(2). This is similar to the construction of spanning surfaces in [Gilmer 1996, §1]. Then

(4-5) qr,F (xi ) ≡ −1 − 4r(2g) ≡ 1 (mod 4). Since B+ is a planar surface, ≡ 1 − β(qr,F ) 2 (n p)(mod 8). We have = 1 + − + − − (4-6) µ(F) 2 n p 2(5 5 ) . We have = 1 − + γ (F) 2 (n p) d 2g. Using (4-4), this becomes

γ (F) = (5+ − 5− − p) g.

Together with (3-1) gives the ﬁrst equation. The last two equations follow from this and Proposition 4.1. Proposition 4.5. Let C be a collection of disjoint ovals in ޒސ(2) with an odd number of components. There is equivalence between: (1) (C, γ, r) is proper for some γ , and r. (2) (C, γ, r) is proper for all possible γ (that is, both g and −g) and for all r. (3) C is even. (4) Every component of B−(C) has even Euler characteristic. 308 PATRICK M. GILMER

Proof. Since C has an odd number of components and each oval represents 2g, we have [C] = 2g ∈ H1(᐀). Thus γ must be either g or −g. Also C is proper for either γ and either r if and only if lk(Ci , C \ Ci ) is even. On the other hand, lk(Ci , C \ Ci ) is even if and only if C is even. The equivalence of the last two conditions is easily seen. Proposition 4.6. Let C be an even collection of disjoint ovals in ޒސ(2), then (C, ±g, r) are proper for both r = r−1/8 and r = r3/8. Moreover ± ≡ + − − − + 1 Arf(C, g, r−1/8) 5 5 n 2 (mod 8), ± ≡ + − − − − 3 Arf(C, g, r3/8) 5 5 n 2 (mod 8). Proof. By Proposition 4.5, every component of B− has even Euler characteristic. Thus we can pick a vector field on B− which is tangent to the boundary and pointed in the direction of the orientation of C with (p − n − 1)/2 zeros of index -2. This defines a surface in ᐀ lying over B− with (p − n − 1)/2 disks removed around the zeros. As above, we complete this surface to form a spanning surface F for L(C) in ᐀ by adding Mobius¨ bands above the removed disks around the zeros. B− is a planar surface disjoint union a Mobius¨ band with some holes removed. The same can be said of the surface we obtain when we delete neighborhoods of the singularities of the vector field. One can see the γ (F) is either g or −g. As we will see, this allows us to compute our Arf invariants without ambiguity. Let us now write γ (F) = ±g, and read plus, if indeed it is plus, and read minus, if indeed it is minus. As in proof of Proposition 4.4, we let xi denote the cores of these Mobius¨ bands. − Equation (4-5) gives us qr,F (xi ). Let α denote an orientation reversing curve in B , then, as in (4-2) and (4-3),

+1 if r = r− ≡ 1 − ≡ 1/8 (4-7) qr,F (α) 2 4r(g) (mod 4). −1 if r = r3/8 Thus +1 if r = r− , ≡ 1 − − + 1/8 β(qr,F ) 2 (p n 1) (mod 8). −1 if r = r3/8, Equation (4-6) gives µ(F). Plugging this into (3-1), we obtain the stated results but where must read ± according to whether γ (F) is ±g. However an application of Proposition 4.1 to both these equations shows that they must hold for the other choice of γ as well.

5. Main results

By the proof of [Gilmer 1996, Theorem 3.1], we have: ARFINVARIANTSOFREALALGEBRAICCURVES 309

Theorem 5.1. If Aޒ is a nodal M-curve of degree 2k, then ᐀(Aޒ) is planar cobordant to ᐀(X2k).

Corollary 5.2. Let Aޒ be a nodal M-curve of degree 2k. ᐀(Aޒ) is homologous to 2kg ∈ H1(᐀). If k is even, Aޒ cannot be weakly equivalent to an even simple curve. If k is odd, Aޒ cannot be weakly equivalent to an odd simple curve. The next theorem follows from Proposition 4.2, the invariance of Arf invariants of proper links under planar cobordism, and Theorem 5.1.

Theorem 5.3. Let Aޒ be a nodal M-curve of degree 2k and suppose (Aޒ, kg, r−1/8) ≡ 1 2 is proper. Then Arf(Aޒ, kg, r−1/8) 2 k (mod 8). We can now give the proof of Theorem 1.1. Theorem 5.3 is more general but its application requires that one calculate Arf(Aޒ, kg, r−1/8).

Proof of Theorem 1.1. Suppose k is even, and Aޒ is weakly equivalent to an odd simple curve. Then, by Proposition 4.3, (Aޒ, kg, r−1/8) is proper. So, by Theorem 5.3, ≡ 1 2 Arf(Aޒ, 0, r−1/8) 2 k (mod 8). By Proposition 4.4, one of (5+ − 5− − p)g or (5+ − 5− − p + 2)g is zero, and + − + − 0 (mod 8) if 5 − 5 − p ≡ 0 (mod 4), Arf(C, 0, r−1/8) ≡ 5 − 5 − p + 2 (mod 8) if 5+ − 5− − p ≡ 2 (mod 4).

Since ᐀(C) and ᐀(Aޒ) are planar cobordant, we have

Arf(Aޒ, 0, r−1/8) ≡ Arf(C, 0, r−1/8)(mod 8). This gives the k even case. Now suppose k is odd, and Aޒ is weakly equivalent to an even simple curve. Then, by Proposition 4.5, (Aޒ, kg, r−1/8) is proper. So, by Theorem 5.3, ≡ 1 2 Arf(Aޒ, kg, r−1/8) 2 k (mod 8). By Proposition 4.6, ≡ + − − − + 1 Arf(C, kg, r−1/8) 5 5 n 2 (mod 8).

Since ᐀(C) and ᐀(Aޒ) are planar cobordant, we have

Arf(Aޒ, kg, r−1/8) ≡ Arf(C, kg, r−1/8)(mod 8).

This gives the k odd case. We now show how Fiedler’s original congruence for certain nonsingular curves is equivalent to Theorem 1.1 when Aޒ is a simple curve, even or odd, as the case may be. 310 PATRICK M. GILMER

Theorem 5.4 (Fiedler). Let Aޒ be a nonsingular M-curve of degree 2k. If k is even, and Aޒ is an odd simple curve, then p − n ≡ −k2 (mod 16).

If k is odd, and Aޒ is an even simple curve, then p − n ≡ 1 (mod 16).

Proof. We have Gudkov’s congruence:

(5-1) p − n ≡ k2 (mod 8).

Harnack’s inequality is extremal: 2k − 1 (5-2) p + n = 1 + = 2k2 − 3k + 2. 2 Adding Equations (5-1) and (5-2) , and dividing by 2: 3k2 − 3k + 2 (5-3) p ≡ (mod 4). 2 According to [Rokhlin 1978, Equation 4], (k − 1)(k − 2) k2 − 3k + 2 (5-4) 5+ − 5− = = . 2 2 Subtracting (5-3) from (5-4), we obtain

(5-5) 5+ − 5− − p ≡ −k2 (mod 4).

At this point, we consider separately three different cases: k ≡ 0 (mod 4), k ≡ 2 (mod 4), and k ≡ 1 (mod 2). Now assume that k ≡ 0 (mod 4) and Aޒ is an odd simple curve. By (5-5), 5+ − 5− − p ≡ 0 (mod 4). Thus by Theorem 1.1 + − − − ≡ 1 2 ≡ 5 5 p 2 k 0 (mod 8). Thus, using (5-4),

2p = k2 − 3k + 2 ≡ −3k + 2 (mod 16).

Subtracting (5-2), p − n = −2k2 ≡ 0 (mod 16), which agrees with the desired conclusion. Now assume that k ≡ 2 (mod 4) and Aޒ is an odd simple curve. We still have + − − − ≡ 1 2 ≡ 5 5 p 0 (mod 4), but now 2 k 2 mod 8. So, by Theorem 1.1, + − − − ≡ 1 2 − ≡ 5 5 p 2 k 2 0 (mod 8). ARFINVARIANTSOFREALALGEBRAICCURVES 311

Using Equations (5-4) and (5-2), as above, we obtain the conclusion of the theorem to be proved. Now assume k ≡ 1 (mod 2) and Aޒ is an even simple curve. By Theorem 1.1, + − − − ≡ 1 2 − ≡ 5 5 n 2 (k 1) 0 (mod 8). Using (5-4) we get −2n ≡ 3k − 3 (mod 16), and ﬁnally, using (5-2),

2 p − n ≡ 2k − 1 ≡ 1 (mod 16).

6. Fiedler’s curves

6.1. Prohibiting a curve of degree 6. Consider the hypothetical curve of degree six prohibited by Fiedler [1986, Figure 1]. We denote this Ᏽ-curve by C1. We note that there is only one orientation on this two component curve up isotopy. So we equip C1 with the orientation which allows the unsmoothing move that we take below. C1 is proper for γ = ±g and r equal either r−1/8 or r3/8. We perform some safe type II moves and type III moves on C1 followed by an unsmoothing move, and a balanced type I move to obtain this Ᏽ-curve:

which we denote by C2. Since the unsmoothing move decreases the number of components, (C2, ±g, r) must be proper, [Gilmer 1993a, Corollary 6.10]. Thus

Arf(C1, ±g, r) ≡ Arf(C2, ±g, r)(mod 8) for either r. − We pick a “vector field” on B (C2) which is tangential to the boundary pointed in the direction of the orientation, has a whole arc of tangent directions at each double point (as in the spanning surface for td in section 3 ) and has a single singularity of index −2. This describes a spanning surface F for L(C2). We = − = −9 calculate that γ (F) g and µ(F) 2 . We have a basis for H1(F, ޚ/2ޚ) consisting of two 1-sided curves δ1, δ2 on F in the affine part of picture, the fiber x over the singularity of index −2, and the line at infinity α. As in (4-1), (4-5), and (4-7) we have respectively = = = = q−1/8,F (δ1) qr−1/8,F (δ2) 1, qr−1/8,K (x) 1, qr−1/8,K (α) 1. 312 PATRICK M. GILMER

≡ Thus β(qr−1/8,F ) 4 (mod 8), and hence ≡ − ≡ + 9 ≡ 1 6≡ 9 Arf(C2, 3g, r−1/8) Arf(C2, g, r−1/8) 4 2 2 2 (mod 8).

By Theorem 5.3, C1 is not a real algebraic curve of degree 6.

6.2. Prohibiting a curve of degree 8. Consider the hypothetical curve of degree eight prohibited by Fiedler [1986, Figure 2]. We denote this Ᏽ-curve by C3 and equip it with the only semiorientation on C3, up to isotopy, which is consistent with [Rokhlin 1978, Equation 4] and Fiedler’s alternation of ovals with respect to a pencil of lines [1982] when applied to C3 smoothed. For this calculation, we use the language of ﬂoppy curves as developed in [Gilmer 1996]. We draw a real ﬂoppy curve C4 in ޒސ(2) whose corresponding link in ᐀ is isotopic to the link that corresponds to C3 with this orientation:

The outer curve with two ﬂops is oriented clockwise. The inner curve with two ﬂops is oriented counterclockwise. Ten of the ovals are oriented counterclockwise and eight clockwise.

We check that C3 is proper for γ = 0 and r = r−1/8. We can extend the vector ﬁeld on the boundary over B+ with ten singularities of index −2. This speciﬁes a spanning surface F. One calculates that γ (F) = 0 and µ(F) = 6. Using Equation ≡ ≡ (4-5), we thus get β(qr−1/8,F ) 10 2 (mod 8). Therefore

Arf(C3, 0, r−1/8) ≡ 2 − 6 ≡ 4 6≡ 0 (mod 8).

By Theorem 5.3, C3 is not a real algebraic curve of degree 8.

References

[Fiedler 1982] T. Fidler, “Pencils of lines and the topology of real algebraic curves”, Izv. Akad. Nauk SSSR Ser. Mat. 46:4 (1982), 853–863. In Russian; translated in Math. USSR Izv. 21 (1983), 161–170. MR 84e:14019 Zbl 0522.14014 [Fiedler 1983] T. Fidler, “New congruences in the topology of real plane algebraic curves”, Dokl. Akad. Nauk SSSR 270:1 (1983), 56–58. In Russian; translated in Sov. Math. Dokl. 27 (1983), 566– 568. MR 85b:14039 Zbl 0541.14022 ARFINVARIANTSOFREALALGEBRAICCURVES 313

[Fiedler 1986] T. Fidler, “New congruences in the topology of singular real algebraic plane curves”, Dokl. Akad. Nauk SSSR 286:5 (1986), 1075–1079. In Russian; translated in Sov. Math. Dokl. 33 (1986), 262–266. MR 87j:14053 Zbl 0608.14025 [Finashin 1996] S. M. Finashin, “On the topology of real plane algebraic curves with nondegenerate quadratic singularities”, Algebra i Analiz 8:6 (1996), 186–204. In Russian; translated in St. Petersburg Math. J. 8:6 (1997) 1039-1051. MR 99c:14067 Zbl 0885.14030 [Gilmer 1992] P. Gilmer, “Real algebraic curves and link cobordism”, Paciﬁc J. Math. 153:1 (1992), 31–69. MR 93c:57005 Zbl 0784.57002 [Gilmer 1993a] P. M. Gilmer, “Link cobordism in rational homology 3-spheres”, J. Knot Theory Ramiﬁcations 2:3 (1993), 285–320. MR 94m:57012 Zbl 0797.57003 [Gilmer 1993b] P. Gilmer, “A method for computing the Arf invariants of links”, pp. 174–181 in Quantum topology, Ser. Knots Everything 3, World Sci., River Edge, NJ, 1993. MR 95d:57003 Zbl 0839.57005 [Gilmer 1996] P. Gilmer, “Real algebraic curves and link cobordism. II”, pp. 73–84 in Topology of real algebraic varieties and related topics, edited by V. Kharlamov et al., Amer. Math. Soc. Transl. Ser. 2 173, Amer. Math. Soc., Providence, RI, 1996. MR 97e:57006 Zbl 0864.57008 [Gilmer 2000] P. M. Gilmer, “Floppy curves, with applications to real algebraic curves”, pp. 39–76 in Real algebraic geometry and ordered structures (Baton Rouge, LA, 1996), edited by C. Delzell and J. Madden, Contemp. Math. 253, Amer. Math. Soc., Providence, RI, 2000. MR 2001j:14079 Zbl 0986.14039 [Gordon and Litherland 1978] C. M. Gordon and R. A. Litherland, “On the signature of a link”, Invent. Math. 47:1 (1978), 53–69. MR 58 #18407 Zbl 0391.57004 [Kharlamov and Viro 1988] V. M. Kharlamov and O. Y. Viro, “Extensions of the Gudkov–Rohlin congruence”, pp. 357–406 in Topology and geometry—Rohlin Seminar, edited by O. Y. Viro, Lec- ture Notes in Math. 1346, Springer, Berlin, 1988. MR 90a:14030 Zbl 0678.14004 [Rokhlin 1978] V. A. Rokhlin, “Complex topological characteristics of real algebraic curves”, Us- pekhi Mat. Nauk 33:5 (1978), 77–89. In Russian; translated in Russian Math. Surveys 33:5 (1978), 85–98. MR 81m:14024 Zbl 0437.14013 [Viro 1978] O. Y. Viro, “Obobwennye neravenstva Petrovskogo i Arnol~da na krivyh s osobennost mi”, Uspekhi Mat. Nauk 33:4 (1978), 145–146. [Viro 1986] O. Y. Viro, “Progress in the topology of real algebraic varieties over the last six years”, Uspekhi Mat. Nauk 41:3 (1986), 45–67. In Russian; translated in Russian Math. Surveys 41:3 (1986), 55–82. MR 87m:14023 Zbl 0619.14015 [Viro and Orevkov 2001] O. Y. Viro and S. Y. Orevkov, “Congruence modulo 8 for real algebraic curves of degree 9”, Uspekhi Mat. Nauk 56:4 (2001), 137–138. In Russian; translated in Russian Math. Surveys 56:4 (2001), 770-771. MR 2002h:14097 Zbl 1049.14045

Received October 5, 2005.

PATRICK M.GILMER DEPARTMENT OF MATHEMATICS LOUISIANA STATE UNIVERSITY BATON ROUGE, LA 70803 UNITED STATES [email protected] www.math.lsu.edu/~gilmer/ PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

ISOMETRIES OF THE QUASIHYPERBOLIC METRIC

PETER HÄSTÖ

We study the curvature and isometries of the quasihyperbolic metric on plane domains. We prove that, except for the trivial case of a half-plane, the isometries are exactly the similarity mappings. We need to assume that the boundary of the domain is C3 smooth.

1. Introduction

Let D ⊂ ޒ2 be an open set and let δ(x) = d(x, ∂ D) be the distance to the boundary. The quasihyperbolic metric in D is the conformal metric with density δ(x)−1; it is given by Z ds(z) kD(x, y) = inf , γ γ δ(z) where the infimum is taken over paths γ connecting x and y in D and ds represents integration with respect to arc-length. The quasihyperbolic metric was first introduced in the 1970s, and since then it has found innumerable applications, especially in the theory of quasiconformal mappings: see [Gehring and Osgood 1979; Gehring and Palka 1976; Herron and Koskela 1996; Martin 1985; Martin and Osgood 1986]. New connections are still being made; for instance P. Jones and S. Smirnov [2000] gave a criterion for removability of a set in the domain of definition of a Sobolev space in terms of the integrability of the quasihyperbolic metric (see also [Koskela and Nieminen 2005]), while Z. Balogh and S. Buckley [2003] used the metric in a geometric characterization of Gromov-hyperbolic spaces. Despite the prominence of the quasihyperbolic metric, there have been almost no investigations of its geometry. Exceptions are [Martin 1985; Martin and Osgood 1986], the second of which was the main motivation for the approach presented in this paper, and H. Linden’s´ [2005] and R. Klen’s´ [2007] theses. Part of the reason for this lack of geometrical investigations is probably that the density of the quasihyperbolic metric is not differentiable in the entire domain, which places the metric outside the standard framework of Riemannian metrics.

MSC2000: 30F45. Keywords: quasihyperbolic metric, isometries, Gaussian curvature. Supported by the Research Council of Norway, Grant 160192/V30.

315 316 PETER HÄSTÖ

At least two modifications of the quasihyperbolic metric have been proposed which do not suffer from this problem. J. Ferrand [1988] suggested replacing the density δ−1 by |a − b| σD(x) = sup . a,b∈∂ D |a − x| |b − x| −1 −1 Note that δ(x) ≤ σD(x) ≤ 2δ(x) , so the Ferrand metric and the quasihyperbolic metric are bilipschitz equivalent. Moreover, the Ferrand metric is Mobius¨ invariant, whereas the quasihyperbolic metric is only Mobius¨ quasi-invariant. A second variant was proposed more recently by R. Kulkarni and U. Pinkall [1994] (see also [Herron et al. 2003]). The K–P metric is defined by the density n 2r o µ (x) = inf : x ∈ B(z, r) ⊂ D . D (r − |x − z|)2 Equivalently, the infimum is taken over the hyperbolic densities of x in balls contained in D. This density satisfies the same estimate as Ferrand’s density, namely −1 −1 δ(x) ≤ µD(x) ≤ 2δ(x) , and the K–P metric is also Mobius¨ invariant. Although the Ferrand and K–P metrics are in some sense better behaved than the quasihyperbolic metric, they suffer from the shortcoming that it is very difficult to get a grip even of the density, even in simple domains. Despite this, D. Herron, Z. Ibragimov and D. Minda [Herron et al. 2006] recently managed to solve the isometry problem for the K–P metric in most cases. By the isometry problem for a metric d we mean the characterization of mappings f : D → ޒ2 with

dD(x, y) = d f (D)( f (x), f (y)) for all x, y ∈ D. Notice that in some sense we are dealing here with two different metrics, due to the dependence on the domain. Hence the usual way of approaching the isometry problem is by looking at some intrinsic features of the metric which are then preserved under the isometry. Since irregularities of the domain, such as cusps, often lead to more distinctive features, this implies that the problem is often easier for more complicated domains. The work just cited bears out this heuristic — the authors were able to show that all isometries of the K–P metric are Mobius¨ mappings except in simply and doubly connected domains. Their proof is based on studying the curvature of the metric. For the quasihyperbolic metric, formulae for the curvature were worked out already in [Martin and Osgood 1986] (see our Section 3), and were used in that paper to prove that all the isometries of the disc are similarity mappings. These will be our main tools in this paper. The other source of the ideas used below are the papers [Hast¨ o¨ and Ibragimov 2005; 2007; Hast¨ o¨ et al. 2006; Hast¨ o¨ and Linden´ 2004] on isometries of some other similarity- and Mobius-invariant¨ metrics. ISOMETRIESOFTHEQUASIHYPERBOLICMETRIC 317

There are three steps in characterizing quasihyperbolic isometries: showing they are conformal, that they are Mobius,¨ and that they are similarities. The ﬁrst step was carried out by Martin and Osgood [1986, Theorem 2.6] for completely arbitrary domains, so there is no more work to do there. In Section 4 we will use their results on the curvature of the quasihyperbolic metric and some new ideas in order to prove that the conformal isometries are Mobius¨ (second step). For this we need to assume that the boundary of the domain is at least C3-smooth. In Section 2 we work on the third step, showing that Mobius¨ isometries are similarities provided the boundary is C1. In Section 3 we study the Gaussian curvature of the quasihyperbolic metric, and the gradient of the curvature.

Notation. If D is a subset of ޒ2, we denote by ∂ D and D its boundary and closure. For x ∈ D ޒ2 we set δ(x) = d(x, ∂ D) = min{|x − z| : z ∈ ∂ D}. We identify ޒ2 with ރ, and speak about the real and imaginary axes, etc. We will often work with a mapping f : D → ޒ2. In such cases we will use a prime to denote quantities on the image side, e.g. x0 = f (x), D0 = f (D) and δ0(x) = d(x, ∂ D0). By B(x, r) we denote a disc with center x and radius r, and by [x, y], (x, y] the closed and half-open segments between x and y. We denote by ޒ2 ∪ {∞} the one-point compactiﬁcation of ޒ2. The cross-ratio |a, b, c, d| for distinct points a, b, c, d ∈ ޒ2 ∪ {∞} is deﬁned by |a − c||b − d| |a, b, c, d| = , |a − b||c − d| with the understanding that |∞−x|/|∞−y| = 1 for all x, y ∈ ޒ2. A homeomor- phism f : ޒ2 ∪ {∞} → ޒ2 ∪ {∞} is a Mobius¨ mapping if

f (a), f (b), f (c), f (d) = |a, b, c, d| for every quadruple of distinct points a, b, c, d in the domain. A mapping of a subdomain of ޒ2 ∪{∞} is Mobius¨ if it is a restriction of a Mobius¨ mapping deﬁned on ޒ2 ∪ {∞}.AMobius¨ mapping can always be decomposed as i ◦ s, where i is an inversion or the identity and s is a similarity. For more information on Mobius¨ mappings see [Beardon 1995, Section 3], for instance.

2. Isometries which are Möbius

Let D be a domain and ζ ∈ ∂ D. We say that ζ is circularly accessible if there exists a disc B ⊂ D such that ζ ∈ ∂ B. Lemma 2.1. Let D ( ޒ2 be a Jordan domain with circularly accessible boundary, and let f : D → ޒ2 by a quasihyperbolic isometry which is also Möbius. Then, up to composition by similarity mappings, f is the identity or the inversion in a circle centered at a boundary point. 318 PETER HÄSTÖ

Proof. Assume that f is not a similarity. Since f is a Mobius¨ map, it is, up to similarities, an inversion. Similarities are always isometries of the quasihyperbolic metric, so it suffices to consider the case when f is an inversion in a unit sphere. Denote the center of this sphere by w. Suppose first that w 6∈ D and let ζ ∈ ∂ D be the closest boundary point to w. We normalize so that ζ lies on the positive real axis and w = 0. Since ζ is circularly accessible, we find a disc B(z, r) ⊂ D containing ζ in its closure. Since ζ is the closest boundary point to w, we see that z has to lie on the positive real axis, as well. Let x and y satisfy ζ < x < y ≤ ζ(ζ +2r)/(ζ+r). The right-hand inequality ensures that ζ is the closest boundary point to [x, y], and that ζ 0 is the closest boundary point to [x0, y0]. Thus we find that 0 0 |x − ζ| 0 0 |x − ζ | k (x, y) = log and k 0 (x , y ) = log . D |y − ζ| D |y0 − ζ 0| Since f is the inversion in the unit sphere, we have |x − ζ| |x0 − ζ 0| = , |x| |ζ| 0 0 and similarly for y. Then the equation exp kD(x, y) = exp kD0 (x , y ) gives us |x − ζ| |x − ζ| |y||ζ| = , |y − ζ| |x||ζ| |y − ζ | i.e., |x| = |y|. This contradiction shows that w ∈ D. Since f maps D into ޒ2, it is clear that w 6∈ D, so w is a boundary point. We call D a Ck domain if ∂ D is locally the graph of a Ck function. Note that if D is a C1 domain, then certainly every boundary point is circularly accessible.

Proposition 2.2. Let D ( ޒ2 be a C1 domain, and let f : D → ޒ2 by a quasihyperbolic isometry which is also Möbius. If D is not a half-plane, then f is a similarity. Proof. Assume that f is not a similarity map. By Lemma 2.1, there is no loss of generality in considering only the case when f is the inversion in a circle centered at a boundary point; moreover, we normalize so that the origin is this center. Let ζ be a boundary point of D distinct from 0 and let u be the inward pointing unit normal at ζ. For all sufﬁciently small t > 0, the point xt = ζ + tu lies in D and its closest boundary point is ζ. For such s < t, we have t k (x , x ) = log . D t s s To estimate the distance of the image points, we use the inequality ISOMETRIESOFTHEQUASIHYPERBOLICMETRIC 319

0 0 0 0 |x − y | 0 0 jD0 (x , y ) = log 1 + ≤ kD0 (x , y ), min{δ0(x0), δ0(y0)} which is always valid (since kD0 is the inner metric of jD0 , e.g. [Gehring and Palka 1976, Lemma 2.1]). We also need the formula |x − y| |x0 − y0| = |x| |y| for the length distortion of an inversion. Using these facts and the estimate δ0(x0) ≤ |x0 − ζ 0|, we derive the inequality 0 0 0 0 |x − y | kD0 (x , y ) ≥ log 1 + min{δ0(x0), δ0(y0)} |x − y|/(|x| |y|) ≥ log 1 + min{|x0 − ζ 0|, |y0 − ζ 0|} |x − y| |ζ| = log 1 + |x| |y| min{|x − ζ|/|x|, |y − ζ|/|y|} |x − y| |ζ| = log 1 + . min{|y| |x − ζ|, |x| |y − ζ|}

Applying this inequality to the points xt and xs as deﬁned before, we have

0 0 (t − s) |ζ| kD0 (xt , xs) ≥ log 1 + . min{t |xt |, s |xs|}

Let us choose t = 2s. Since |x2s| and |xs| both tend to |ζ| as s → 0, we see that the second term in the minimum is smaller. Since the inversion is supposed to be an isometry, we can use the formula for kD(xt , xs) from before with the previous inequality to conclude that 2s (2s − s) |ζ| log ≥ log 1 + . s s |xs|

Taking the exponential function gives |xs| ≥ |ζ|. Since xs = ζ + su, this implies that hζ − 0, ui ≤ 0 as s → 0, where h , i denotes the scalar product. Applying the same argument, but starting with points on the image side, we conclude that the opposite inequality is also valid. (There is actually a slight asymmetry here: the domain D0 need not have circularly accessible boundary at the origin. However, it is clear that this does not affect the argument so far.) Thus it follows that hζ − 0, ui = 0 for all boundary points. But since the boundary is 1 assumed to be C , this implies that the domain is a half-plane. From [Martin and Osgood 1986, Theorem 2.8] we know that if f : D → ޒ2 is a quasihyperbolic isometry, then f is conformal in D. In dimensions three and 320 PETER HÄSTÖ higher every conformal mapping is Mobius.¨ It is easy to see that the proofs in this section work also in the higher dimensional case. Therefore, we have proved: Corollary 2.3. Let D be a C1 domains in ޒn, n ≥ 3, which is not a half-space. Then every quasihyperbolic isometry is a similarity mapping. Example 2.4. If we do not assume the boundary is C1, there are some other domains with nontrivial isometries, namely the punctured plane and sector domains (those whose boundary consists of two rays). In these cases, inversions centered at the puncture or the vertex of the sector are isometries. The previous proposition strongly suggests that there are no further examples.

3. Curvature of the quasihyperbolic metric

Let D be a domain in ޒ2. We call a disc B ⊂ D maximal, if it is not contained in any other disc contained in D. The set consisting of the centers of all maximal discs in D is called the medial axis of D and denoted by MA(D). The medial axis and differentiability properties of the distance-to-the-boundary function have been studied in [Caffarelli and Friedman 1979; Choi et al. 1997; Damon 2003]. In a general domain the Gaussian curvature of the quasihyperbolic metric is not defined, since the distance-to-the-boundary function is not C2. M. Heins [1962] considered this situation for a quite general class of metric, and defined the notions of upper and lower curvature. Martin and Osgood [1986, Section 3] worked with these curvatures in the context of the quasihyperbolic metric. However, if our domain is sufficiently regular (say C2), and we are considering points not on the medial axis, then the upper and lower curvature agree, and define the curvature. In this case the curvature of kD is given by 2 ᏷D(z) = −δ(z) 4 log δ(z); see [Heins 1962, (1.3)] or [Martin and Osgood 1986, (3.1)]. On the medial axis this formula does not make sense, but the upper and lower curvatures still agree, and both equal −∞, by [Martin and Osgood 1986, Corollary 3.12]. The next lemma is a specialization of [Martin and Osgood 1986, Lemma 3.5] to the case there the upper and lower curvatures agree. Lemma 3.1. Let G and G˜ be C2 domains such that B(z, r) ⊂ G ∩ G˜ and ζ ∈ (∂G)∩(∂G˜ )∩∂ B(z, r). If there is a neighborhood U of ζ such that G ∩U ⊂ G˜ ∩U ˜ ˜ and d(z, ∂G \ U) > d(z, ∂G), then ᏷G(z) ≤ ᏷G˜ (z). Using this lemma we can derive the following very plausible statement, which says that the Gaussian curvature of the quasihyperbolic metric depends only on the curvature of the boundary at the closest boundary point. We sill need some more notation. ISOMETRIESOFTHEQUASIHYPERBOLICMETRIC 321

Let B be a disc with ζ ∈ (∂ B) ∩ (∂ D). Then we call B the osculating disc at ζ if ∂ B and ∂ D have second order contact at ζ. Let D be at least a C2 domain. Then there exists an osculating disc at every boundary point ζ. If this disc has radius r, then we deﬁne Rζ to be r if the disc lies in the direction of the interior of D, and k−2 k −r otherwise. Note that the function ζ 7→ 1/Rζ is C in a C domain, k ≥ 2.

Proposition 3.2. Let D ( ޒ2 be a C2 domain and z ∈ D \ MA(D) have closest boundary point ζ ∈ ∂ D. Then

Rζ 1 ᏷D(z) = − = − . Rζ − δ(z) 1 − δ(z)/Rζ

If z lies on the medial axis, then ᏷D(z) = −∞. Proof. The medial axis consists of points equidistant to two or more nearest boundary points, and of centers of osculating circles. For the former, the claim that ᏷D(z) = −∞ follows from [Martin and Osgood 1986, Corollary 3.12]. So we assume that z has a unique nearest boundary point, ζ. We suppose further that Rζ > 0, the other case begin similar. Let B(w, Rζ ) be w−ζ the osculating disc at ζ. We deﬁne Bt = B(w + t, Rζ + t), and note that ∂ Bt Rζ contains ζ for all t > −Rζ . We have the formula r r ᏷B r (x) = − = − (0, ) |x| r − d(x, ∂ B(0, r)) for the curvature of the quasihyperbolic metric in a ball [Martin and Osgood 1986,

Lemma 3.7], so we can calculate ᏷Bt (z) explicitly. ˜ Using the previous lemma with G = D and G = Bt for t > 0 gives ᏷D(z) ≤

᏷Bt (z). If z is the center of B0, then right-hand-side of the this inequality tends to −∞ as t → 0, which completes the proof of the claim regarding the medial axis. So we assume that z is not the center of B0, and then we can apply the Lemma 3.1 = ˜ = ≤ with G Bt for t < 0 (sufﬁciently close to 0) and G D to get ᏷Bt (z) ᏷D(z). Thus we have ≤ ≤ ᏷B−t (z) ᏷D(z) ᏷Bt (z) = for small t > 0. Since ᏷Bt is continuous in t, we get ᏷D(z) ᏷B0 (z) as we let t → 0. The proof is completed by applying the aforementioned formula for the curvature to the ball B0 = B(w, Rζ ). Let f : D → ޒ2 be a C1 mapping. By ∇ f we denote the gradient of f , i.e. the ˜ vector (∂1 f, ∂2 f ), and by ∇ f (z) we denote δ(z)∇ f (z). The reason for multiplying by δ(x) is that |x − y| δ(y) = lim , x→y kD(x, y) 322 PETER HÄSTÖ so that the ∇˜ operator is more natural in the setting where the quasihyperbolic but not the Euclidean distance is preserved (see (3-1), below). ˜ We next present an explicit formula for ∇᏷D. For this we need a mapping which associates to every point in D \ MA(D) its closest boundary point. We call this mapping ζ = ζ(z).

Lemma 3.3. Let D ( ޒ2 be a C3 domain. Then ˜ ∇᏷D(z) = (᏷D(z) + 1) ᏷D(z)∇δ(z) − (᏷D(z) + 1)∇ Rζ(z) for every z off the medial axis, where all differentiation is with respect to the variable z.

Proof. We use the formula from Proposition 3.2. Thus

2 1 δ(z) ᏷D(z) ∇᏷ (z) = −∇ = ᏷ (z)2∇ = (R ∇δ(z) − δ(z)∇ R ), D − D 2 ζ ζ 1 δ(z)/Rζ Rζ Rζ

1 where we understand ζ as a function of z. Note that Rζ and δ are C , since D is C3 and we are not on the medial axis. From Proposition 3.2 we also get

δ(z) ᏷ (z) + 1 = D . Rζ ᏷D(z) Thus we continue the equation by

˜ 2 δ(z) δ(z) ∇᏷D(z) = ᏷D(z) ∇δ(z) − ∇ Rζ Rζ Rζ = (᏷D(z) + 1) ᏷D(z)∇δ(z) − (᏷D(z) + 1)∇ Rζ .

We next show that |∇˜ ᏷| is an intrinsic quantity of the quasihyperbolic metric.

Lemma 3.4. Let D be a C3 domain. If f : D → ޒ2 is a quasihyperbolic isometry, ˜ ˜ then |∇᏷D(z)| = |∇᏷ f (D)( f (z))| for every z ∈ D.

Proof. We know that f is conformal. For a unit vector u we ﬁnd that

˜ ᏷D(z + εu) − ᏷D(z) (3-1) ∇᏷D(z), u = lim ε→0 kD(z + εu, z) ᏷ ( f (z + εu)) − ᏷ ( f (z)) = lim f (D) f (D) . ε→0 k f (D)( f (z + εu), f (z))

Next we note that f (z +εu) = f (z)+εf 0(z)u + O(ε2). Here f 0(z)u is understood as complex multiplication. Now deﬁne another unit vector u˜ = ( f 0(z)/| f 0(z)|)u. ISOMETRIESOFTHEQUASIHYPERBOLICMETRIC 323

We continue the previous equation by ᏷ ( f (z) + εf 0(z)u) − ᏷ ( f (z)) ∇˜ = f (D) f (D) ᏷D(z), u lim 0 ε→0 k f (D)( f (z) + εf (z)u, f (z)) | 0 |h∇ ˜i ε f (z) ᏷ f (D)( f (z)), u ˜ = lim = ∇᏷ f (D)( f (z)), u˜ . ε→0 ε| f 0(z)|δ0( f (z))−1 ˜ ˜ Since u was an arbitrary unit vector, we get |∇᏷D(z)| = |∇᏷ f (D)( f (z))|.

4. Isometries

We know that similarities are always quasihyperbolic isometries, and we want to show that in most cases these are the only ones. In view of the results in Section 2, it sufﬁces for us to show that a quasihyperbolic isometry is a Mobius¨ mapping, so this will be what we aim at in the proofs of this section. A curve γ in D is a (quasihyperbolic) geodesic if

kD(x, y) = kD(x, z) + kD(z, y) for all x, z, y ∈ γ in this order. It is clear from this definition that geodesics are preserved by isometries. A geodesic ray is a geodesic which is isometric to ޒ+. For every z ∈ D we easily find one geodesic ray, namely [z, ζ(z)), which also happens to be a Euclidean line segment. The idea is to show that this geodesic is somehow special (from a quasihyperbolic point-of-view), so that it would map to a geodesic ray of the same kind. 2 2 Lemma 4.1. If D (ޒ be a C domain with a boundary point ξ such that 1/Rξ =0, every isometry f : D → ޒ2 of the quasihyperbolic metric is Möbius. Proof. Let B ⊂ D be a nonmaximal disc whose boundary contains ξ and let z denote the center of B. By Proposition 3.2 we find that ᏷D ≡ −1 on the segment = [ ≡ − 0 0 = 0 γ z, ξ) Thus ᏷ f (D) 1 on γ , so 1/Rζ 0(z0) 0 for every point z on this curve. We consider two cases: either ζ 0(z0) is just a single point for all z0 ∈ γ 0, or it sweeps out a nondegenerate subcurve of the boundary ∂ D0 as z0 varies over γ 0. (There is no third possibility, since ζ 0 is a continuous function on γ 0.) In the single-point case we see that γ 0 has to be a line segment, since the boundary does not have corners. In this case we find that 0 0 |x − ξ| 0 0 |x − ξ | kD(x, y) = log and kD0 (x , y ) = log , |y − ξ| |y0 − ξ 0| where ξ 0 is the closest boundary point to the every point on γ 0. But this easily implies that f is Mobius¨ on γ . Since f is conformal it follows by uniqueness of analytic extension that f is a Mobius¨ mapping on all of D. So we consider the second case, that ζ 0(z0) sweeps out a nondegenerate subcurve 324 PETER HÄSTÖ of the boundary ∂ D0. Since the curvature of the boundary at all these points is zero, it follows that the piece of the boundary is a line segment, L0. Let U 0 ⊂ D0 be an open set such that (∂U 0)∩(∂ D0)= L0 and the nearest boundary point of every point in U 0 lies in L0. The geometry of the quasihyperbolic metric 0 in U is the same as in a half-plane; in particular ᏷D0 ≡ −1 on U . Then ᏷D ≡ −1 on U = f −1(U 0), so it follows that (∂U) ∩ (∂ D) = L, for some line segment L. So it follows that f |U is the restriction of a quasihyperbolic isometry of the half- plane. But these are only the Mobius¨ mappings. Then we again conclude from the uniqueness of analytic extension that f is a Mobius¨ mapping on all of D. We call a domain strictly concave if its complement is strictly convex. Corollary 4.2. If D ( ޒ2 is a C2 domain which is not a half-plane, strictly convex or strictly concave, every quasihyperbolic isometry is a similarity mapping.

Proof. Suppose that 1/Rζ 6= 0 for all boundary points. Since 1/Rζ is continuous by assumption, this implies that it is either everywhere positive, or everywhere negative. In these cases we have a strictly convex and strictly concave domain, respectively, which was ruled out by assumption. So we find some point at which 1/Rζ = 0. Then it follows from Lemma 4.1 that the isometry is Mobius¨ and from Proposition 2.2 that it is a similarity. So we are left with only two types of domains that we cannot handle: strictly convex and strictly concave ones. As usual when working with isometries, the nicest domains turn out to be the most difficult. Unfortunately, we need to assume more regularity of the boundary in order to take care of these cases. Theorem 4.3. Let D ( ޒ2 be a C3 domain, which is not a half-plane. Then every isometry f : D → ޒ2 of the quasihyperbolic metric is a similarity mapping. Proof. In view of Corollary 4.2, we may restrict ourselves to the case when ᏷D(z) 6= −1 for all z ∈ D. Let z ∈ D \MA(D) and ζ be its nearest boundary point. We note that ∇δ(z) and ∇ Rζ are perpendicular – first of all, ∇δ(z) is parallel to z − ζ; second, Rζ is a constant in the direction of z − ζ, since ζ is the closest boundary point to all points on this line (near z). If D is bounded, clearly Rζ has a critical point. If D is unbounded, 1/Rζ cannot have any other limit than 0 at ∞ (although a limit need not exist, of course). Thus Rζ has a critical point in the unbounded case as well. Let ζ be a critical point of ξ 7→ Rξ and fix a point z ∈ D with ᏷D(z) 6= −∞ whose nearest boundary point is ζ. Of course, ∇ Rζ = 0 at the critical point ζ. Then it follows from Lemma 3.3 that ˜ ∇᏷D(z) = (᏷D(z) + 1)᏷D(z)∇δ(z). 0 Since the curvature is intrinsic to the metric, we have ᏷D0 (z ) = ᏷D(z). Also, ˜ 0 ˜ |∇᏷D0 (z )| = |∇᏷D(z)| by Lemma 3.4, so we have ISOMETRIESOFTHEQUASIHYPERBOLICMETRIC 325

0 0 0 (᏷D(z) + 1)᏷D(z)∇δ(z) = (᏷D(z) + 1) ᏷D(z)∇δ (z ) − (᏷D(z) + 1)∇ Rζ 0(z0) 6= − ∇ 0 0 ∇ 0 We know that ᏷D(z) 1 and that δ (z ) and Rζ 0(z0) are orthogonal. Thus the previous equation simpliﬁes to 2 0 0 2 0 2 ᏷D(z)|∇δ(z)| = ᏷D(z)|∇δ (z )| + (᏷D(z) + 1) ∇ Rζ 0(z0) . Since |∇δ| = 1 off the medial axis for every domain, this equation implies that ∇ 0 = Rζ 0 0. 0 So for our point z, ∇᏷D(z) and ∇᏷D0 (z ) point to the nearest boundary point of z and z0, respectively. Let γ = [z, ζ ). Note that γ is a geodesic of the quasihyperbolic metric. Also, ∇᏷D(z) and γ are parallel at z. Now γ maps to some geodesic ray 0 0 0 0 0 0 γ , and since f is a conformal mapping, γ is parallel to ∇᏷D0 (z ) at z . But [z , ζ ) 0 0 is a geodesic parallel to ∇᏷D0 (z ) at z , and since geodesics are unique (when the density is C2, i.e. except possibly on the medial axis) we see that γ 0 = [z0, ζ 0). So we have shown that f ([z, ζ )) = [z0, ζ 0). Moreover, we have 0 0 |x − ζ| 0 0 |x − ζ | kD(x, y) = log and kD0 (x , y ) = log |y − ζ| |y0 − ζ 0| for x, y ∈ [z, ζ ). Thus we see that f is just a similarity on [z, ζ ). But f is a conformal map, so this implies that f is a similarity in all of D. Acknowledgment. I would like to thank Zair Ibragimov for several discussions about the isometries of this and related metrics.

References

[Balogh and Buckley 2003] Z. M. Balogh and S. M. Buckley, “Geometric characterizations of Gro- mov hyperbolicity”, Invent. Math. 153:2 (2003), 261–301. MR 2004i:30042 Zbl 1059.30038 [Beardon 1995] A. F. Beardon, The geometry of discrete groups, Graduate Texts in Mathematics 91, Springer, New York, 1995. MR 97d:22011 Zbl 0528.30001 [Caffarelli and Friedman 1979] L. A. Caffarelli and A. Friedman, “The free boundary for elastic- plastic torsion problems”, Trans. Amer. Math. Soc. 252 (1979), 65–97. MR 80i:35059 Zbl 0426. 35033 [Choi et al. 1997] H. I. Choi, S. W. Choi, and H. P. Moon, “Mathematical theory of medial axis transform”, Paciﬁc J. Math. 181:1 (1997), 57–88. MR 99m:53008 Zbl 0885.53004 [Damon 2003] J. Damon, “Smoothness and geometry of boundaries associated to skeletal structures, I: Sufﬁcient conditions for smoothness”, Ann. Inst. Fourier (Grenoble) 53:6 (2003), 1941–1985. MR 2005d:58009 Zbl 1047.57014 [Ferrand 1988] J. Ferrand, “A characterization of quasiconformal mappings by the behaviour of a function of three points”, pp. 110–123 in Complex analysis (Joensuu, 1987), edited by I. Laine et al., Lecture Notes in Math. 1351, Springer, Berlin, 1988. MR 89m:30040 Zbl 0661.30015 [Gehring and Osgood 1979] F. W. Gehring and B. G. Osgood, “Uniform domains and the quasihyperbolic metric”, J. Analyse Math. 36 (1979), 50–74. MR 81k:30023 Zbl 0449.30012 [Gehring and Palka 1976] F. W. Gehring and B. P. Palka, “Quasiconformally homogeneous domains”, J. Analyse Math. 30 (1976), 172–199. MR 55 #10676 Zbl 0349.30019 326 PETER HÄSTÖ

[Hästö and Ibragimov 2005] P. Hästö and Z. Ibragimov, “Apollonian isometries of planar domains are Möbius mappings”, J. Geom. Anal. 15:2 (2005), 229–237. MR 2006d:30063 Zbl 1081.30039 [Hästö and Ibragimov 2007] P. Hästö and Z. Ibragimov, “Apollonian isometries of regular domains are Möbius mappings”, Ann. Acad. Sci. Fenn. Math. 32 (2007), 83–98. [Hästö and Lindén 2004] P. Hästö and H. Lindén, “Isometries of the half-Apollonian metric”, Com- plex Var. Theory Appl. 49:6 (2004), 405–415. MR 2005b:51031 Zbl 1063.30042 [Hästö et al. 2006] P. Hästö, Z. Ibragimov, and H. Lindén, “Isometries of relative metrics”, Comput. Methods Funct. Theory 6:1 (2006), 15–28. MR 2007d:30030 Zbl 1102.30046 [Heins 1962] M. Heins, “On a class of conformal metrics”, Nagoya Math. J. 21 (1962), 1–60. MR 26 #1451 Zbl 0113.05603 [Herron and Koskela 1996] D. A. Herron and P. Koskela, “Conformal capacity and the quasihyperbolic metric”, Indiana Univ. Math. J. 45:2 (1996), 333–359. MR 97i:30060 Zbl 0862.30021 [Herron et al. 2003] D. A. Herron, W. Ma, and D. Minda, “A Möbius invariant metric for regions on the Riemann sphere”, pp. 101–118 in Future trends in geometric function theory (Jyväskylä, 2003), edited by D. Herron, Rep. Univ. Jyväskylä Dep. Math. Stat. 92, Univ. Jyväskylä, 2003. MR 2058115 Zbl 1058.30020 [Herron et al. 2006] D. Herron, Z. Ibragimov, and D. Minda, “Geodesics and curvature of Möbius invariant metrics”, Preprint, 2006, Available at http://math.uc.edu/~herron/papers/gcmim.pdf. [Jones and Smirnov 2000] P. W. Jones and S. K. Smirnov, “Removability theorems for Sobolev functions and quasiconformal maps”, Ark. Mat. 38 (2000), 263–279. MR 2001i:46049 Zbl 1034.30014 [Klén 2007] R. Klén, Local convexity properties of metric balls, Ph.L. thesis, University of Turku, Finland, 2007. [Koskela and Nieminen 2005] P. Koskela and T. Nieminen, “Quasiconformal removability and the quasihyperbolic metric”, Indiana Univ. Math. J. 54:1 (2005), 143–151. MR 2005k:46079 Zbl 1078.30016 [Kulkarni and Pinkall 1994] R. S. Kulkarni and U. Pinkall, “A canonical metric for Möbius structures and its applications”, Math. Z. 216:1 (1994), 89–129. MR 95b:53017 Zbl 0813.53022 [Lindén 2005] H. Lindén, “Quasihyperbolic geodesics and uniformity in elementary domains”, Ann. Acad. Sci. Fenn. Math. Diss. 146 (2005). Dissertation, University of Helsinki, 2005. MR 2183008 Zbl 02232563 [Martin 1985] G. J. Martin, “Quasiconformal and bi-Lipschitz homeomorphisms, uniform domains and the quasihyperbolic metric”, Trans. Amer. Math. Soc. 292:1 (1985), 169–191. MR 87a:30037 Zbl 0584.30020 [Martin and Osgood 1986] G. J. Martin and B. G. Osgood, “The quasihyperbolic metric and associated estimates on the hyperbolic metric”, J. Analyse Math. 47 (1986), 37–53. MR 88e:30060 Zbl 0621.30023

Received August 26, 2005.

PETER HAST¨ O¨ DEPARTMENT OF MATHEMATICAL SCIENCES P.O. BOX 3000 FI-90014 UNIVERSITYOF OULU FINLAND peter.hasto@helsinki.ﬁ http://cc.oulu.ﬁ/~phasto PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

THE TWO-PARAMETER QUANTUM GROUP OF EXCEPTIONAL TYPE G2 AND LUSZTIG’S SYMMETRIES

NAIHONG HUAND QIAN SHI

We give the defining structure of the two-parameter quantum group of type G2 defined over a field ޑ(r, s) (with r 6= s), and prove the Drinfel’d double structure as its upper and lower triangular parts, extending a result of Benkart and Witherspoon in type A and a recent result of Bergeron, Gao, and Hu in types B, C, D. We further discuss Lusztig’s ޑ-isomorphisms

from Ur,s(G2) to its associated object Us−1,r−1 (G2), which give rise to the usual Lusztig symmetries defined not only on Uq (G2) but also on the cen- c −1 tralized quantum group Uq (G2) only when r = s = q. (This also reflects the distinguishing difference between our newly defined two-parameter object and the standard Drinfel’d–Jimbo quantum groups.) Some interesting (r, s)-identities holding in Ur,s(G2) are derived from this discussion.

1. The two-parameter quantum group Ur,s(G2) Let ދ = ޑ(r, s) be a field of rational functions with two indeterminates r, s. Let 8 be a finite root system of G2 with 5 a base of simple roots, which is 3 a subset of a Euclidean space E = ޒ with an inner product (,). Let 1, 2, 3 denote an orthonormal basis of E. Then 5 = {α1 = 1 − 2, α2 = 2 + 3 − 21} and 8 = ±{α1, α2, α2 + α1, α2 + 2α1, α2 + 3α1, 2α2 + 3α1}. In this case, we set (α1,α1)/2 (α2,α2)/2 3 (α ,α )/2 (α ,α )/2 3 r1 = r = r, r2 = r = r and s1 = s 1 1 = s, s2 = s 2 2 = s . We begin by defining the two-parameter quantum group of type G2, which is new.

Deﬁnition 1.1. Let U = Ur,s(G2) be the associative algebra over ޑ(r, s) generated ±1 0±1 by the symbols ei , fi , ωi and ωi (1 ≤ i ≤ 2), subject to the relations ±1 ±1 ±1 0±1 0±1 0±1 −1 0 0−1 (G1) [ ωi , ωj ] = [ ωi , ωj ] = [ ωi , ωj ] = 0, ωi ωi = 1 = ωj ωj ;

MSC2000: primary 17B37, 81R50; secondary 17B35. Keywords: 2-parameter quantum group, Hopf skew-dual pairing, Hopf dual pairing, Drinfel’d quantum double, Lusztig symmetries. Hu is supported in part by the NNSF (Grant 10431040), the PCSIRT, the TRAPOYT and the FUDP from the MOE of China, the SRSTP from the STCSM, and the Shanghai Priority Academic Disci- pline from the SMEC.

327 328 NAIHONG HU AND QIANSHI

−1 = −1 −1 = −1 ω1 e1 ω1 (rs ) e1, ω1 f1 ω1 (r s) f1, ω e ω−1 = s3 e , ω f ω−1 = s−3 f , (G2) 1 2 1 2 1 2 1 2 −1 = −3 −1 = 3 ω2 e1 ω2 r e1, ω2 f1 ω2 r f1, −1 = 3 −3 −1 = −3 3 ; ω2 e2 ω2 (r s ) e2, ω2 f2 ω2 (r s ) f2 0 0−1 = −1 0 0−1 = −1 ω1 e1 ω1 (r s) e1, ω1 f1 ω1 (rs ) f1, ω0 e ω0−1 = r 3 e , ω0 f ω0−1 = r −3 f , (G3) 1 2 1 2 1 2 1 2 0 0−1 = −3 0 0−1 = 3 ω2 e1 ω2 s e1, ω2 f1 ω2 s f1, 0 0−1 = −3 3 0 0−1 = 3 −3 ; ω2 e2 ω2 (r s ) e2, ω2 f2 ω2 (r s ) f2 0 ωi − ωi (G4) [ ei , fj ] = δi j for 1 ≤ i, j ≤ 2; ri − si (G5) ((r, s)-Serre relations in positive part)

2 −3 −3 −3 2 e2e1 − (r + s ) e2e1e2 + (rs) e1e2 = 0, 4 2 2 3 2 2 2 2 2 2 e1e2 − (r + s)(r + s ) e1e2e1 + rs(r + s )(r + rs + s ) e1e2e1 3 2 2 3 6 4 −(rs) (r + s)(r + s ) e1e2e1 + (rs) e2e1 = 0; (G6) ((r, s)-Serre relations in negative part)

2 −3 −3 −3 2 f1 f2 − (r + s ) f2 f1 f2 + (rs) f2 f1 = 0, 4 2 2 3 2 2 2 2 2 2 f2 f1 − (r + s)(r + s ) f1 f2 f1 + rs(r + s )(r + rs + s ) f1 f2 f1 3 2 2 3 6 4 −(rs) (r + s)(r + s ) f1 f2 f1 + (rs) f1 f2 = 0.

Proposition 1.2. The algebra Ur,s(G2) is a Hopf algebra with comultiplication, counit and antipode given by

±1 ±1 ±1 1(ωi ) = ωi ⊗ ωi , 1(ei ) = ei ⊗ 1 + ωi ⊗ ei , 0 ±1 0 ±1 0 ±1 = ⊗ + ⊗ 0 1(ωi ) = ωi ⊗ ωi , 1( fi ) 1 fi fi ωi , ± 0 ±1 ±1 = ∓1 = − −1 ε(ωi ) = ε(ωi ) = 1, S(ωi ) ωi , S(ei ) ωi ei , 0 ±1 0 ∓1 0 −1 ε(ei ) = ε( fi ) = 0, S(ωi ) = ωi , S( fi ) = − fi ωi .

−1 Remark 1.3. (I) When r = q = s , the quotient Hopf algebra of Ur,s(G2) 0 −1 modulo the Hopf ideal generated by elements ωi − ωi (1 ≤ i ≤ 2) is just the standard quantum group Uq (G2) of Drinfel’d–Jimbo type; the quotient modulo 0 −1 the Hopf ideal generated by elements ωi − zi ωi (1 ≤ i ≤ 2), where zi runs over c the center, is the centralized quantum group Uq (G2). (II) In any Hopf algebra Ᏼ, there exist left-adjoint and right-adjoint actions deﬁned by the Hopf algebra structure: THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 329 X X adl a (b) = a(1) b S(a(2)), adr a (b) = S(a(1)) b a(2), (a) (a) P where 1(a) = (a) a(1) ⊗ a(2) ∈ Ᏼ ⊗ Ᏼ, for any a, b ∈ Ᏼ. From the viewpoint of adjoint actions, the (r, s)-Serre relations (G5) and (G6) take on the simpler forms

1−ai j (adl ei ) (ej ) = 0 for any i 6= j,

1−ai j (adr fi ) ( fj ) = 0 for any i 6= j.

2. The Drinfel’d quantum double

Deﬁnition 2.1. A (Hopf) dual pairing of two Hopf algebras Ꮽ and ᐁ (see [Bergeron et al. 2006] or [Klimyk and Schmudgen¨ 1997]) is a bilinear form h , i : ᐁ×Ꮽ → ދ such that

(1) h f, 1Ꮽi = εᐁ( f ), h1ᐁ, ai = εᏭ(a),

(2) h f, a1a2i = h4ᐁ( f ), a1 ⊗ a2i, h f1 f2, ai = h f1 ⊗ f2, 4Ꮽ(a)i, for all f, f1, f2 ∈ ᐁ, and a, a1, a2 ∈ Ꮽ, where εᐁ, εᏭ denote the counits of ᐁ, Ꮽ and 4ᐁ, 4Ꮽ the comultiplications. A direct consequence of the deﬁning properties above is that

hSᐁ( f ), ai = h f, SᏭ(a)i, f ∈ ᐁ, a ∈ Ꮽ, where Sᐁ, SᏭ denote the respective antipodes of ᐁ and Ꮽ. Definition 2.2. A bilinear form h , i : ᐁ × Ꮽ → ދ is called a skew-dual pairing of two Hopf algebras Ꮽ and ᐁ (see [Bergeron et al. 2006]) if h , i : ᐁcop × Ꮽ → ދ is a Hopf dual pairing of Ꮽ and ᐁcop, where ᐁcop is the Hopf algebra having the −1 cop = opposite comultiplication to ᐁ, and Sᐁ Sᐁ is invertible. ±1 Denote by Ꮾ = B(G2) the Hopf subalgebra of Ur,s(G2) generated by ej , ωj , 0 0 0±1 and by Ꮾ = B (G2) the one generated by fj , ωj , where j = 1, 2. Proposition 2.3. There exists a unique skew-dual pairing h , i : Ꮾ0 × Ꮾ → ޑ(r, s) of the Hopf subalgebras Ꮾ and Ꮾ0 such that 1 (3) h fi , ej i = δi j (1 ≤ i, j ≤ 2), si −ri 0 −1 0 −3 hω1, ω1i = rs , hω1, ω2i = r , (4) 0 3 0 3 −3 hω2, ω1i = s , hω2, ω2i = r s , 0±1 −1 0±1 −1 0 ∓1 (5) hωi , ωj i = hωi , ωj i = hωi , ωj i (1 ≤ i, j ≤ 2), 330 NAIHONG HU AND QIANSHI and all other pairs of generators yield 0. Furthermore, hS(a), S(b)i = ha, bi for a ∈ Ꮾ0, b ∈ Ꮾ. Proof. Since any skew-dual pairing of bialgebras is determined by its values on generators, uniqueness is clear. We proceed to prove the existence of the pairing. We begin by defining a bilinear form h , i : Ꮾ0cop × Ꮾ → ޑ(r, s) first on the generators satisfying (3), (4), and (5). Then we extend it to a bilinear form on 0cop op × 4 0cop = 4 Ꮾ Ꮾ by requiring that (1) and (2) hold for Ꮾ Ꮾ0 . We will verify that the relations in Ꮾ and Ꮾ0 are preserved, ensuring that the form is well-defined and so is a dual pairing of Ꮾ and Ꮾ0cop by definition. It is direct to check that the bilinear form preserves all the relations among ±1 0±1 0 the ωi in Ꮾ and the ωi in Ꮾ . Next, the structure constants (4) ensure the compatibility of the form defined above with those relations of (G2) and (G3) in Ꮾ or Ꮾ0, respectively. We are left to verify that the form preserves the (r, s)-Serre relations in Ꮾ and Ꮾ0. It suffices to show that the form on Ꮾ0cop × Ꮾ preserves the (r, s)-Serre relations in Ꮾ; the verification for Ꮾ0cop is similar. First, let us show that the form preserves the (r, s)-Serre relation of degree 2 in Ꮾ, that is, 2 −3 −3 −3 −3 2 hX, e2e1 − (r + s ) e2e1e2 + r s e1e2 i = 0, where X is any word in the generators of Ꮾ0. It suffices to consider three mono- = 2 2 mials: X f2 f1, f2 f1 f2, f1 f2 . However, in the degree 2’s situation for type G2, its proof is the same as that of type C2 (see [Bergeron et al. 2006, (7C) and thereafter]). Next, we verify that the (r, s)-Serre relation of degree 4 in Ꮾ is preserved by the form; that is, we show that

4 2 2 3 2 2 2 2 2 2 X, e1e2 − (r + s)(r + s ) e1e2e1 + rs(r + s )(r + rs + s ) e1e2e1 3 2 2 3 6 4 −(rs) (r + s)(r + s ) e1e2e1 + (rs) e2e1 vanishes, where X is any word in the generators of Ꮾ0. By deﬁnition, this expression equals

(4) 2 2 (6) 4 (X), e1 ⊗e1 ⊗e1 ⊗e1 ⊗e2 −(r +s)(r +s ) e1 ⊗e1 ⊗e1 ⊗e2 ⊗e1 2 2 2 2 +rs(r +s )(r +rs+s ) e1 ⊗e1 ⊗e2 ⊗e1 ⊗e1 3 2 2 6 −(rs) (r +s)(r +s ) e1 ⊗e2 ⊗e1 ⊗e1 ⊗e1 +(rs) e2 ⊗e1 ⊗e1 ⊗e1 ⊗e1 , 4 h i 4op where in the left-hand side of the pairing , indicates Ꮾ0 . In order for any one of these terms to be nonzero, X must involve exactly four f1 factors, one f2 0±1 factor, and arbitrarily many ωj factors ( j = 1, 2). It sufﬁces to consider ﬁve key cases: THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 331

= 4 (i) If X f1 f2, we have

(4) 0 0 0 0 0 0 0 4 (X) = ω1 ⊗ ω1 ⊗ ω1 ⊗ ω1 ⊗ f1 + ω1 ⊗ ω1 ⊗ ω1 ⊗ f1 ⊗ 1 0 0 0 4 + ω1 ⊗ω1 ⊗ f1 ⊗1⊗1+ω1 ⊗ f1 ⊗1⊗1⊗1+ f1 ⊗1⊗1⊗1⊗1 0 0 0 0 0 0 0 · ω2 ⊗ ω2 ⊗ ω2 ⊗ ω2 ⊗ f2 + ω2 ⊗ ω2 ⊗ ω2 ⊗ f2 ⊗ 1 0 0 0 + ω2 ⊗ω2 ⊗ f2 ⊗1⊗1+ω2 ⊗ f2 ⊗1⊗1⊗1+ f2 ⊗1⊗1⊗1⊗1 .

Expanding 4(4)(X), we get 120 relevant terms having a nonzero contribution to (6). They are listed in Table 1, together with their pairing values, where we have

(4) 4 Table 1. Terms of 1 ( f1 f2) in (6) and their pairing values. We write β instead of ω0 and · instead of ⊗ to save space. We have = h i4h i = h 0 i ¯ = h 0 i also set a f1, e1 f2, e2 , x ω1, ω1 , x ω1, ω2 .

Summand in (6) 1 Summand in (6) 2 3 · 2 · · · 3 · 2 · · · ¯ f1β1 β2 f1β1 β2 f1β1β2 f1β2 f2 a f1β1 β2 f1β1 β2 f1β1β2 β1 f2 f1 xa 2 · 2 · · · 2 · 2 · · · ¯ β1 f1β1 β2 f1β1 β2 f1β1β2 f1β2 f2 xa β1 f1β1 β2 f1β1 β2 f1β1β2 β1 f2 f1 xxa 2 · 2 · · · 2 2 · 2 · · · ¯ 2 β1 f1β1β2 f1β1 β2 f1β1β2 f1β2 f2 x a β1 f1β1β2 f1β1 β2 f1β1β2 β1 f2 f1 xx a 3 · 2 · · · 3 3 · 2 · · · ¯ 3 β1 f1β2 f1β1 β2 f1β1β2 f1β2 f2 x a β1 f1β2 f1β1 β2 f1β1β2 β1 f2 f1 xx a 3 · · · · 3 · · · · ¯ f1β1 β2 β1 f1β1β2 f1β1β2 f1β2 f2 xa f1β1 β2 β1 f1β1β2 f1β1β2 β1 f2 f1 xxa 2 · · · · 2 2 · · · · ¯ 2 β1 f1β1 β2 β1 f1β1β2 f1β1β2 f1β2 f2 x a β1 f1β1 β2 β1 f1β1β2 f1β1β2 β1 f2 f1 xx a 2 · · · · 3 2 · · · · ¯ 3 β1 f1β1β2 β1 f1β1β2 f1β1β2 f1β2 f2 x a β1 f1β1β2 β1 f1β1β2 f1β1β2 β1 f2 f1 xx a 3 · · · · 4 3 · · · · ¯ 4 β1 f1β2 β1 f1β1β2 f1β1β2 f1β2 f2 x a β1 f1β2 β1 f1β1β2 f1β1β2 β1 f2 f1 xx a 3 · 2 · · · 2 3 · 2 · · · ¯ 2 f1β1 β2 β1 f1β2 f1β1β2 f1β2 f2 x a f1β1 β2 β1 f1β2 f1β1β2 β1 f2 f1 xx a 2 · 2 · · · 3 2 · 2 · · · ¯ 3 β1 f1β1 β2 β1 f1β2 f1β1β2 f1β2 f2 x a β1 f1β1 β2 β1 f1β2 f1β1β2 β1 f2 f1 xx a 2 · 2 · · · 4 2 · 2 · · · ¯ 4 β1 f1β1β2 β1 f1β2 f1β1β2 f1β2 f2 x a β1 f1β1β2 β1 f1β2 f1β1β2 β1 f2 f1 xx a 3 · 2 · · · 5 3 · 2 · · · ¯ 5 β1 f1β2 β1 f1β2 f1β1β2 f1β2 f2 x a β1 f1β2 β1 f1β2 f1β1β2 β1 f2 f1 xx a 3 · 2 · · · 3 · 2 · · · ¯ f1β1 β2 f1β1 β2 β1 f1β2 f1β2 f2 xa f1β1 β2 f1β1 β2 β1 f1β2 β1 f2 f1 xxa 2 · 2 · · · 2 2 · 2 · · · ¯ 2 β1 f1β1 β2 f1β1 β2 β1 f1β2 f1β2 f2 x a β1 f1β1 β2 f1β1 β2 β1 f1β2 β1 f2 f1 xx a 2 · 2 · · · 3 2 · 2 · · · ¯ 3 β1 f1β1β2 f1β1 β2 β1 f1β2 f1β2 f2 x a β1 f1β1β2 f1β1 β2 β1 f1β2 β1 f2 f1 xx a 3 · 2 · · · 4 3 · 2 · · · ¯ 4 β1 f1β2 f1β1 β2 β1 f1β2 f1β2 f2 x a β1 f1β2 f1β1 β2 β1 f1β2 β1 f2 f1 xx a 3 · · · · 2 3 · · · · ¯ 2 f1β1 β2 β1 f1β1β2 β1 f1β2 f1β2 f2 x a f1β1 β2 β1 f1β1β2 β1 f1β2 β1 f2 f1 xx a 2 · · · · 3 2 · · · · ¯ 3 β1 f1β1 β2 β1 f1β1β2 β1 f1β2 f1β2 f2 x a β1 f1β1 β2 β1 f1β1β2 β1 f1β2 β1 f2 f1 xx a 2 · · · · 4 2 · · · · ¯ 4 β1 f1β1β2 β1 f1β1β2 β1 f1β2 f1β2 f2 x a β1 f1β1β2 β1 f1β1β2 β1 f1β2 β1 f2 f1 xx a 3 · · · · 5 3 · · · · ¯ 5 β1 f1β2 β1 f1β1β2 β1 f1β2 f1β2 f2 x a β1 f1β2 β1 f1β1β2 β1 f1β2 β1 f2 f1 xx a 3 · 2 · · · 3 3 · 2 · · · ¯ 3 f1β1 β2 β1 f1β2 β1 f1β2 f1β2 f2 x a f1β1 β2 β1 f1β2 β1 f1β2 β1 f2 f1 xx a 2 · 2 · · · 4 2 · 2 · · · ¯ 4 β1 f1β1 β2 β1 f1β2 β1 f1β2 f1β2 f2 x a β1 f1β1 β2 β1 f1β2 β1 f1β2 β1 f2 f1 xx a 2 · 2 · · · 5 2 · 2 · · · ¯ 5 β1 f1β1β2 β1 f1β2 β1 f1β2 f1β2 f2 x a β1 f1β1β2 β1 f1β2 β1 f1β2 β1 f2 f1 xx a 3 · 2 · · · 6 3 · 2 · · · ¯ 6 β1 f1β2 β1 f1β2 β1 f1β2 f1β2 f2 x a β1 f1β2 β1 f1β2 β1 f1β2 β1 f2 f1 xx a 332 NAIHONG HU AND QIANSHI

Table 1. (Continued)

Summand in (6) 3 Summand in (6) 4 3 · 2 · 2 · · ¯2 3 · 3 · 2 · · ¯3 f1β1 β2 f1β1 β2 β1 f2 β1 f1 f1 x xa f1β1 β2 β1 f2 f1β1 β1 f1 f1 x xa 2 · 2 · 2 · · ¯2 2 2 · 3 · 2 · · ¯3 2 β1 f1β1 β2 f1β1 β2 β1 f2 β1 f1 f1 x x a β1 f1β1 β2 β1 f2 f1β1 β1 f1 f1 x x a 2 · 2 · 2 · · ¯2 3 2 · 3 · 2 · · ¯3 3 β1 f1β1β2 f1β1 β2 β1 f2 β1 f1 f1 x x a β1 f1β1β2 β1 f2 f1β1 β1 f1 f1 x x a 3 · 2 · 2 · · ¯2 4 3 · 3 · 2 · · ¯3 4 β1 f1β2 f1β1 β2 β1 f2 β1 f1 f1 x x a β1 f1β2 β1 f2 f1β1 β1 f1 f1 x x a 3 · · 2 · · ¯2 2 3 · 3 · · · ¯3 2 f1β1 β2 β1 f1β1β2 β1 f2 β1 f1 f1 x x a f1β1 β2 β1 f2 β1 f1β1 β1 f1 f1 x x a 2 · · 2 · · ¯2 3 2 · 3 · · · ¯3 3 β1 f1β1 β2 β1 f1β1β2 β1 f2 β1 f1 f1 x x a β1 f1β1 β2 β1 f2 β1 f1β1 β1 f1 f1 x x a 2 · · 2 · · ¯2 4 2 · 3 · · · ¯3 4 β1 f1β1β2 β1 f1β1β2 β1 f2 β1 f1 f1 x x a β1 f1β1β2 β1 f2 β1 f1β1 β1 f1 f1 x x a 3 · · 2 · · ¯2 5 3 · 3 · · · ¯3 5 β1 f1β2 β1 f1β1β2 β1 f2 β1 f1 f1 x x a β1 f1β2 β1 f2 β1 f1β1 β1 f1 f1 x x a 3 · 2 · 2 · · ¯2 3 3 · 3 · 2 · · ¯3 3 f1β1 β2 β1 f1β2 β1 f2 β1 f1 f1 x x a f1β1 β2 β1 f2 β1 f1 β1 f1 f1 x x a 2 · 2 · 2 · · ¯2 4 2 · 3 · 2 · · ¯3 4 β1 f1β1 β2 β1 f1β2 β1 f2 β1 f1 f1 x x a β1 f1β1 β2 β1 f2 β1 f1 β1 f1 f1 x x a 2 · 2 · 2 · · ¯2 5 2 · 3 · 2 · · ¯3 5 β1 f1β1β2 β1 f1β2 β1 f2 β1 f1 f1 x x a β1 f1β1β2 β1 f2 β1 f1β1 β1 f1 f1 x x a 3 · 2 · 2 · · ¯2 6 3 · 3 · 2 · · ¯3 6 β1 f1β2 β1 f1β2 β1 f2 β1 f1 f1 x x a β1 f1β2 β1 f2 β1 f1 β1 f1 f1 x x a 3 · 2 · 2 · · ¯2 3 · 3 · 2 · · ¯3 f1β1 β2 f1β1 β2 β1 f2 f1β1 f1 x a f1β1 β2 β1 f2 f1β1 f1β1 f1 x a 2 · 2 · 2 · · ¯2 2 · 3 · 2 · · ¯3 β1 f1β1 β2 f1β1 β2 β1 f2 f1β1 f1 x xa β1 f1β1 β2 β1 f2 f1β1 f1β1 f1 x xa 2 · 2 · 2 · · ¯2 2 2 · 3 · 2 · · ¯3 2 β1 f1β1β2 f1β1 β2 β1 f2 f1β1 f1 x x a β1 f1β1β2 β1 f2 f1β1 f1β1 f1 x x a 3 · 2 · 2 · · ¯2 3 3 · 3 · 2 · · ¯3 3 β1 f1β2 f1β1 β2 β1 f2 f1β1 f1 x x a β1 f1β2 β1 f2 f1β1 f1β1 f1 x x a 3 · · 2 · · ¯2 3 · 3 · · · ¯3 f1β1 β2 β1 f1β1β2 β1 f2 f1β1 f1 x xa f1β1 β2 β1 f2 β1 f1β1 f1β1 f1 x xa 2 · · 2 · · ¯2 2 2 · 3 · · · ¯3 2 β1 f1β1 β2 β1 f1β1β2 β1 f2 f1β1 f1 x x a β1 f1β1 β2 β1 f2 β1 f1β1 f1β1 f1 x x a 2 · · 2 · · ¯2 3 2 · 3 · · · ¯3 3 β1 f1β1β2 β1 f1β1β2 β1 f2 f1β1 f1 x x a β1 f1β1β2 β1 f2 β1 f1β1 f1β1 f1 x x a 3 · · 2 · · ¯2 4 3 · 3 · · · ¯3 4 β1 f1β2 β1 f1β1β2 β1 f2 f1β1 f1 x x a β1 f1β2 β1 f2 β1 f1β1 f1β1 f1 x x a 3 · 2 · 2 · · ¯2 2 3 · 3 · 2 · · ¯3 2 f1β1 β2 β1 f1β2 β1 f2 f1β1 f1 x x a f1β1 β2 β1 f2 β1 f1 f1β1 f1 x x a 2 · 2 · 2 · · ¯2 3 2 · 3 · 2 · · ¯3 3 β1 f1β1 β2 β1 f1β2 β1 f2 f1β1 f1 x x a β1 f1β1 β2 β1 f2 β1 f1 f1β1 f1 x x a 2 · 2 · 2 · · ¯2 4 2 · 3 · 2 · · ¯3 4 β1 f1β1β2 β1 f1β2 β1 f2 f1β1 f1 x x a β1 f1β1β2 β1 f2 β1 f1β1 f1β1 f1 x x a 3 · 2 · 2 · · ¯2 5 3 · 3 · 2 · · ¯3 5 β1 f1β2 β1 f1β2 β1 f2 f1β1 f1 x x a β1 f1β2 β1 f2 β1 f1 f1β1 f1 x x a

Summand in (6) 5 Summand in (6) 5 4 · 3 · 2 · · ¯4 4 · 3 · 2 · · ¯4 β1 f2 f1β1 f1β1 β1 f1 f1 x xa β1 f2 f1β1 f1β1 f1β1 f1 x a 4 · 2 · 2 · · ¯4 2 4 · 2 · 2 · · ¯4 β1 f2 β1 f1β1 f1β1 β1 f1 f1 x x a β1 f2 β1 f1β1 f1β1 f1β1 f1 x xa 4 · 2 · 2 · · ¯4 3 4 · 2 · 2 · · ¯4 2 β1 f2 β1 f1β1 f1β1 β1 f1 f1 x x a β1 f2 β1 f1β1 f1β1 f1β1 f1 x x a 4 · 3 · 2 · · ¯4 4 4 · 3 · 2 · · ¯4 3 β1 f2 β1 f1 f1β1 β1 f1 f1 x x a β1 f2 β1 f1 f1β1 f1β1 f1 x x a 4 · 3 · · · ¯4 2 4 · 3 · · · ¯4 β1 f2 f1β1 β1 f1β1 β1 f1 f1 x x a β1 f2 f1β1 β1 f1β1 f1β1 f1 x xa 4 · 2 · · · ¯4 3 4 · 2 · · · ¯4 2 β1 f2 β1 f1β1 β1 f1β1 β1 f1 f1 x x a β1 f2 β1 f1β1 β1 f1β1 f1β1 f1 x x a 4 · 2 · · · ¯4 4 4 · 2 · · · ¯4 3 β1 f2 β1 f1β1 β1 f1β1 β1 f1 f1 x x a β1 f2 β1 f1β1 β1 f1β1 f1β1 f1 x x a 4 · 3 · · · ¯4 5 4 · 3 · · · ¯4 4 β1 f2 β1 f1 β1 f1β1 β1 f1 f1 x x a β1 f2 β1 f1 β1 f1β1 f1β1 f1 x x a 4 · 3 · 2 · · ¯4 3 4 · 3 · 2 · · ¯4 2 β1 f2 f1β1 β1 f1 β1 f1 f1 x x a β1 f2 f1β1 β1 f1 f1β1 f1 x x a 4 · 2 · 2 · · ¯4 4 4 · 2 · 2 ·0 · ¯4 3 β1 f2 β1 f1β1 β1 f1 β1 f1 f1 x x a β1 f2 β1 f1β1 β1 f1 f1β1 f1 x x a 4 · 2 · 2 · · ¯4 5 4 · 2 · 2 · · ¯4 4 β1 f2 β1 f1β1 β1 f1 β1 f1 f1 x x a β1 f2 β1 f1β1 β1 f1 f1β1 f1 x x a 4 · 3 · 2 · · ¯4 6 4 · 3 · 2 · · ¯4 5 β1 f2 β1 f1 β1 f1 β1 f1 f1 x x a β1 f2 β1 f1 β1 f1 f1β1 f1 x x a THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 333 introduced 4 0 0 a = h f1, e1i h f2, e2i, x = hω1, ω1i, x¯ = hω1, ω2i. The expression in (6) equals (sum of expressions in part 1 of Table 1) − (sum of expressions in part 2) · (r + s)(r 2 + s2) + (sum of expressions in part 3) · rs(r 2 + s2)(r 2 + rs + s2) − (sum of expressions in part 4) · (rs)3(r + s)(r 2 + s2) + (sum of expressions in part 5) · (rs)6. Thus, if we sum up all the pairing values listed in each part of Table 1 and multiply by the appropriate factor, we obtain the pairing value of (6): a(1 + 3x + 5x2 + 6x3 + 5x4 + 3x5 + x6) · 1 − (r + s)(r 2 + s2)x¯ + rs(r 2 + s2) · (r 2 + rs + s2)x¯2 − (rs)3(r + s)(r 2 + s2)x¯3 + (rs)6x¯4 = a(1 + 3x + 5x2 + 6x3 + 5x4 + 3x5 + x6)(1−r 3x¯)(1−r 2sx¯)(1−rs2x¯)(1−s3x¯) = ¯ = h 0 i = −3 0 (because x ω1, ω2 r ). = 4 (ii) X f2 f1 . = 2 2 (iii) X f1 f2 f1 . = 3 (iv) X f1 f2 f1. = 3 (v) X f1 f2 f1 . These four other cases are handled similarly, using the formulas given in the Ap- pendix of [Hu and Shi 2006]. This takes care of 1(4)(X). The proof is completed 0cop by checking that the relations in B are preserved for G2. Deﬁnition 2.4. For any two Hopf algebras Ꮽ and ᐁ connected by a skew-dual pairing h , i, one may form the Drinfel’d quantum double Ᏸ(Ꮽ, ᐁ) as in [Klimyk and Schmudgen¨ 1997, 3.2], which is a Hopf algebra whose underlying coalgebra is Ꮽ ⊗ ᐁ with the tensor product coalgebra structure, whose algebra structure is deﬁned by 0 0 X 0 0 0 0 (7) (a ⊗ f )(a ⊗ f ) = h᏿ᐁ( f(1)), a(1)ih( f(3)), a(3)iaa(2) ⊗ f(2) f for a, a0 ∈ Ꮽ and f, f 0 ∈ ᐁ, and whose antipode S is given by

(8) S(a ⊗ f ) = (1 ⊗ ᏿ᐁ( f ))(᏿Ꮽ(a) ⊗ 1). Clearly, both mappings Ꮽ 3 a 7→ a ⊗ 1 ∈ Ᏸ(Ꮽ, ᐁ) and ᐁ 3 f 7→ 1 ⊗ f ∈ Ᏸ(Ꮽ, ᐁ) are injective Hopf algebra homomorphisms. Denote the image a ⊗ 1 of a in Ᏸ(Ꮽ, ᐁ) by aˆ, and the image 1⊗ f of f by fˆ. By (7), we have the following 334 NAIHONG HU AND QIANSHI cross relations between elements aˆ (for a ∈ Ꮽ) and fˆ (for f ∈ ᐁ) in the algebra Ᏸ(Ꮽ, ᐁ): ˆ X ˆ (9) f aˆ = h᏿ᐁ( f(1)), a(1)ih( f(3)), a(3)iˆa(2) f(2), X ˆ X ˆ (10) h f(1), a(1)i f(2)aˆ(2) = aˆ(1) f(1)h f(2), a(2)i.

In fact, as an algebra the double Ᏸ(Ꮽ, ᐁ) is the universal algebra generated by the algebras Ꮽ and ᐁ with cross relations (9) or, equivalently, (10).

Theorem 2.5. The two-parameter quantum group Ur,s(G2) is isomorphic to the Drinfel’d quantum double Ᏸ(Ꮾ, Ꮾ0).

The proof is the same as that of [Bergeron et al. 2006, Theorem 2.5].

Remark 2.6. The proofs of Proposition 2.3 and Theorem 2.5 show the compatibility of the deﬁning relations of Ur,s(G2). The proof of Theorem 2.5 indicates that the cross relations between Ꮾ and Ꮾ0 are precisely half the ones appearing in (G1)–(G4), and the proof of Proposition 2.3 then shows the compatibility of the remaining relations appearing in Ꮾ and Ꮾ0, including the other half of (G1)–(G4) and the (r, s)-Serre relations (G5)–(G6).

3. Lusztig’s symmetries from Ur,s(G2) to Us−1,r−1 (G2) As we did in [Bergeron et al. 2006] for the classical types A, B, C, D, we call (Us−1,r −1 (G2), h | i) the quantum group associated to (Ur,s(G2), h , i), where the 0 −1 −1 pairing hωi | ωj i is deﬁned by replacing (r, s) with (s , r ) in the deﬁning for- 0 mula for hωi , ωj i. Obviously, 0 0 hωi | ωj i = hωj , ωi i.

We now study Lusztig’s symmetry property between (Ur,s(G2), h , i) and its associated object (Us−1,r −1 (G2), h | i), which indeed indicates the difference in structures between the two-parameter quantum group introduced above and the usual one-parameter quantum group of Drinfel’d–Jimbo type. To deﬁne the Lusztig symmetries, we introduce the notation of divided-power elements (in (Us−1,r −1 (G2), h | i) ). For any nonnegative integer k ∈ ގ, set

s−k − r −k hki = i i hki ! = h i h i · · · hki i −1 −1 , i 1 i 2 i i , si − ri and for any element ei , fi ∈ (Us−1,r −1 (G2), h | i), deﬁne the divided-power elements

(k) k (k) k ei = ei /hkii ! , fi = fi /hkii ! . THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 335

Deﬁnition 3.1. To every i (i = 1, 2), there corresponds a ޑ-linear mapping ᐀i : −1 −1 (Ur,s(G2), h , i) → (Us−1,r −1 (G2), h | i) such that ᐀i (r) = s , ᐀i (s) = r , which 0 acts on the generators ωj , ωj , ej , fj (1 ≤ j ≤ 2) as

−ai j 0 0 0 −ai j ᐀i (ωj ) = ωj ωi , ᐀i (ωj ) = ωj ωi , 0 −1 −1 ᐀i (ei ) = − ωi fi , ᐀i ( fi ) = −(ri si ) ei ωi , and for i 6= j,

−ai j X ν − − 0 − 0 ν + (ν) (−ai j −ν) ν 2 ( ai j ν) ν 2 (1 ai j ) ᐀i (ej ) = (−1) (rs) hωj , ωi i hωi , ωi i ei ej ei , ν=0

−ai j + X ν − − 0 0 − ν + (−ai j −ν) (ν) δi j ν 2 ( ai j ν) ν 2 (1 ai j ) ᐀i ( fj ) = (rj sj ) (−1) (rs) hωi , ωj i hωi , ωi i fi fj fi , ν=0 where (ai j ) is the Cartan matrix of the simple Lie algebra g of type G2, and for any i 6= j, ( 2, if i < j and a 6= 0, δ+ = i j i j 1, otherwise .

Lemma 3.2. ᐀i (i = 1, 2) preserves the deﬁning relations (G1)–(G3) of Ur,s(G2) into its associated object Us−1,r −1 (G2).

Proof. For G2, we have

0 −1 0 0 −3 0 hω1, ω1i = rs = hω1| ω1i, hω1, ω2i = r = hω2| ω1i, 0 3 0 0 3 −3 0 hω2, ω1i = s = hω1| ω2i, hω2, ω2i = r s = hω2| ω2i.

We show that ᐀1, ᐀2 preserve the defining relations (G1)–(G3). (G1) are auto- matically satisfied. To check (G2) and (G3): first of all, by direct calculation, we 0 0 0 0 have ᐀k(hωi , ωj i) = h᐀k(ωi ), ᐀k(ωj )i = hωj , ωi i = hωi | ωj i, for i, j, k ∈ {1, 2}. This fact ensures that ᐀k(k = 1, 2) preserve (G2) and (G3), that is,

−1 0 ᐀k(ωj )᐀k(ei )᐀k(ωj ) = hωi | ωj i᐀k(ei ), −1 0 −1 ᐀k(ωj )᐀k( fi )᐀k(ωj ) = hωi | ωj i ᐀k( fi ), 0 0 −1 0 −1 ᐀k(ωj )᐀k(ei )᐀k(ωj ) = hωj | ωi i ᐀k(ei ), 0 0 −1 0 ᐀k(ωj )᐀k( fi )᐀k(ωj ) = hωj | ωi i᐀k( fi ), where checking the other three identities is equivalent to checking the ﬁrst one.

Lemma 3.3. ᐀i (i = 1, 2) preserves the deﬁning relations (G4) into its associated object Us−1,r −1 (G2). 336 NAIHONG HU AND QIANSHI

Proof. Put 1 = r 2 + rs + s2. To check (G4): for i = 1, 2, we have

0−1 −1 [᐀i (ei ), ᐀i ( fi )] = (ri si )ωi ( fi ei − ei fi )ωi = ᐀i ([ei , fi ]), 3 3 [᐀2(e1), ᐀2( f1)] = [e1e2 − r e2e1, rs( f2 f1 − s f1 f2)] 3 = rs f2[e1, f1]e2 + e1[e2, f2] f1 − r ([e2, f2] f1e1 + e2 f2[e1, f1]) 3 3 − s ([e1, f1] f2e2 + e1 f1[e2, f2]) + (rs) (e2[e1, f1] f2 + f1[e2, f2]e1) 0 0 0 ω2ω1 − ω ω ᐀2(ω1) − ᐀2(ω ) = 2 1 = 1 = ᐀ ([e , f ]), s−1 − r −1 s−1 − r −1 2 1 1 and as for r 3s3 [᐀ (e ), ᐀ ( f )] = (rs2)3e e3 − rs31e e e2 + s1e2e e − e3e , 1 2 1 2 (r + s)212 2 1 1 2 1 1 2 1 1 2 2 3 3 3 2 2 3 (r s) f1 f2 − sr 1f1 f2 f1 + r1f1 f2 f1 − f2 f1 , we have to show that the bracket on the right-hand side is equal to

ω ω3 − ω0 ω03 1(r + s)2 2 1 2 1 . r − s To do so, we introduce the notations of “quantum root vectors” in terms of adjoint actions, as follows:

3 E12 = (adl e1)(e2) = e1e2 − s e2e1, 3 F12 = (adr f1)( f2) = f2 f1 − r f1 f2, 2 2 E112 = (adl e1) (e2) = e1 E12 − rs E12e1, 2 2 F112 = (adr f1) ( f2) = F12 f1 − r s f1 F12, 3 3 2 3 2 2 3 3 E1112 = (adl e1) (e2) = e1e2 − s1e1e2e1 + rs 1e1e2e1 − (rs ) e2e1, 3 3 2 3 2 2 3 3 F1112 = (adr f1) ( f2) = f2 f1 − r1f1 f2 f1 + sr 1f1 f2 f1 − (r s) f1 f2. That is, we need to verify that ω ω3 − ω0 ω03 [ E , F ] = 1(r + s)2 2 1 2 1 . 1112 1112 r − s By direct calculation using the Leibniz rule, we have

0 [e1, F12] = −1ω1 f2, [e2, F12] = f1ω2, 0 [E12, f1] = −1e2ω1, [E12, f2] = ω2e1, ω ω − ω0 ω0 [E , F ] = 1 2 1 2 , 12 12 r − s THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 337

2 2 2 2 0 [e1, F112] = −(r + s) ω1 F12, [e2, F112] = s(s − r ) f1 ω2, 2 0 2 2 2 [E112, f1] = −(r + s) E12ω1, [E112, f2] = r(r − s )ω2e1, 2 2 0 0 [E112, F12] = (r + s) ω1ω2e1, [E12, F112] = (r + s) f1ω1ω2, ω2ω − ω02ω0 [E , F ] = (r + s)2 1 2 1 2 , 112 112 r − s as well as 2 [e1, F1112] = [e1, F112 f1 − rs f1 F112] = −1ω1 F112, 2 [E112, F1112] = [E112, F112 f1 − rs f1 F112] 2 = [E112, F112] f1 − rs f1[E112, F112] + F112[E112, f1] 2 − rs [E112, f1]F112 2 02 0 = 1(r + s) f1ω1 ω2, 2 [E1112, F1112] = [e1 E112 − r sE112e1, F1112] 2 = [e1, F1112]E112 − r sE112[e1, F1112] + e1[E112, F1112] 2 − r s[E112, F1112]e1 2 02 0 = 1ω1[E112, F112] + 1(r + s) [e1, f1]ω1 ω2 ω ω3 − ω0 ω03 = 1(r + s)2 2 1 2 1 . r − s

Thus, we arrive at [᐀1(e2), ᐀1( f2)] = ᐀1([e2, f2]) ∈ Us−1,r −1 (G2).

Lemma 3.4. ᐀2 preserves the (r, s)-Serre relations (G5)1, (G6)1 into its associated object Us−1,r −1 (G2): 2 3 3 3 2 (11) ᐀2(e2) ᐀2(e1) − (r +s )᐀2(e2)᐀2(e1)᐀2(e2) + (rs) ᐀2(e1)᐀2(e2) = 0, 2 3 3 3 2 (12) ᐀2( f1)᐀2( f2) − (r +s )᐀2( f2)᐀2( f1)᐀2( f2) + (rs) ᐀2( f1)᐀2( f2) = 0.

Proof. For the degree 2 (r, s)-Serre relation (G5)1 2 −3 −3 −3 −3 2 e2e1 − (r + s )e2e1e2 + r s e1e2 = 0, observe that

−3 −3 3 (13) ᐀2(e1)᐀2(e2) = r ᐀2(e2)᐀2(e1) − r e1, ᐀2(e2)e1 = s e1᐀2(e2).

Making ᐀2 act algebraically on the left-hand side of (G5)1, we have 2 3 3 3 2 ᐀2(e2) ᐀2(e1) − (r + s )᐀2(e2)᐀2(e1)᐀2(e2) + (rs) ᐀2(e1)᐀2(e2) 3 −3 3 3 = ᐀2(e2)r (᐀2(e1)᐀2(e2) + r e1) − (r + s )᐀2(e2)᐀2(e1)᐀2(e2) 3 −3 −3 + (rs) (r ᐀2(e2)᐀2(e1) − r e1)᐀2(e2) = 0, 338 NAIHONG HU AND QIANSHI proving (11). The proof of (12) is similar.

To prove that ᐀1 preserves the Serre relations, we need three auxiliary lemmas. Lemma 3.5. In the notation in Lemma 3.3, we have

3 [E1112 E112 − r E112 E1112, f2] = 0.

3 4 Proof. Since e1 E1112 − r E1112e1 = adl (e1) (e2) = 0 (Serre relation), and 2 2 [E1112, f2] = [e1 E112 − r sE112e1, f2] = e1[E112, f2] − r s[E112, f2]e1 3 2 2 3 = r (r−s)(r −s ) ω2e1, we obtain

3 [E1112 E112 − r E112 E1112, f2] 3 = E1112[E112, f2] + [E1112, f2]E112 − r E112[E1112, f2] + [E112, f2]E1112 3 2 2 3 2 3 3 = r (r−s)(r −s ) ω2 e1 E112 − r1e1 E1112e1 − (r s) E112e1 3 2 2 3 2 3 2 2 3 3 = r (r−s)(r −s ) ω2 e1 E112 − r1e1 E112e1 + r s1e1 E112e1 − (r s) E112e1 3 2 2 2 = r (r−s)(r −s ) ω2 e1 · (᏿᏾) − rs (᏿᏾) · e1 = 0, where (᏿᏾) denotes the left-hand-side presentation of the (r, s)-Serre relation (G5)2 2 2 5 2 e1 E112 − r (r + s)e1 E112e1 + r sE112e1 = 0, 2 and we used the replacement E1112 = e1 E112 − r sE112e1 in the third equality. Lemma 3.6. In the notation of Lemma 3.3, we have

3 [E1112 E112 − r E112 E1112, f1] = 0. [ ] = − 0 Proof. It is easy to check that E1112, f1 1E112ω1. Thus 3 [E1112 E112 − r E112 E1112, f1] 3 = E1112[E112, f1] + [E1112, f1]E112 − r E112[E1112, f1] + [E112, f1]E1112 3 2 0 = (r + s) (r + s)((rs) E12 E1112 − E1112 E12) + r(r − s)1E112 ω1. It sufﬁces to show that

3 −1 2 (14) E1112 E12 = (rs) E12 E1112 + r(r−s)(r+s) 1E112.

At ﬁrst, we note that the (r, s)-Serre relation (G5)1 is equivalent to

3 E12e2 = r e2 E12. THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 339

3 Since e1e2 = E12 + s e2e1, we get 2 3 2 E112e2 = (e1 E12 − rs E12e1)e2 = r e1e2 E12 − rs E12e1e2 3 3 2 3 = r (E12 + s e2e1)E12 − rs E12(E12 + s e2e1) 2 2 2 3 2 = r(r −s )E12 + (rs) e2(e1 E12 − rs E12e1) 2 2 2 3 = r(r −s )E12 + (rs) e2 E112. Next, we claim

2 3 2 2 2 2 2 E1112e2 = (rs ) e2 E1112 − r(rs−r +s )E112 E12 + (rs) (r +rs−s )E12 E112. 2 2 Indeed, since E1112 = e1 E112 −r sE112e1, E112 = e1 E12 −rs E12e1, and e1e2 = 3 E12 + s e2e1, we have 2 E1112e2 = e1(E112e2) − r sE112(e1e2) 2 2 2 3 2 = r(r − s )e1 E12 + (rs) (e1e2)E112 − r sE112(e1e2) 2 2 2 3 2 3 2 = r(r − s )e1 E12 + (rs) E12 E112 + (rs ) e2e1 E112 − r sE112 E12 2 2 − (rs ) (E112e2)e1 2 2 2 2 2 3 = r(r − s )E112 E12 + (rs) (r − s )E12e1 E12 + (rs) E12 E112 2 3 2 3 4 2 2 2 5 7 + (rs ) e2e1 E112 − r sE112 E12 − r s (r −s )E12e1 − r s e2 E112e1 2 3 2 2 2 2 2 = (rs ) e2 E1112 − r(rs−r +s )E112 E12 + (rs) (r +rs−s )E12 E112. To prove (14), we ﬁrst note that

3 2 [(r + s)((rs) E12 E1112 − E1112 E12) + r(r − s)1E112, f1] 3 = (r + s)(rs) E12[E1112, f1] + [E12, f1]E1112 − (r + s) E1112[E12, f1] + [E1112, f1]E12

+ r(r − s)1(E112[E112, f1] + [E112, f1]E112) 3 3 0 = −(r + s)(rs) 1 E12 E112 + s e2 E1112 ω1 2 0 + (r + s)1 E1112e2 + r sE112 E12 ω1 2 2 0 − r(r − s)(r + s) 1 E112 E12 + rs E12 E112 ω1, which vanishes by the preceding identity. A similar but longer computation (see [Hu and Shi 2006] for details) shows that the bracket

3 2 [(r + s)((rs) E12 E1112 − E1112 E12) + r(r − s)1E112, f2]. also vanishes. Then, through an argument similar to the one used in the deduction of [Benkart et al. 2006, Lemma 3.4], we get (14). By [Benkart et al. 2006, Lemma 3.4], Lemmas 3.5 and 3.6 imply: 340 NAIHONG HU AND QIANSHI

3 Lemma 3.7. E1112 E112 − r E112 E1112 = 0.

Lemma 3.8. ᐀1 preserves the (r, s)-Serre relations (G5)1, (G6)1 into its associated object Us−1,r −1 (G2):

2 3 3 3 2 (15) ᐀1(e2) ᐀1(e1) − (r + s )᐀1(e2)᐀1(e1)᐀1(e2) + (rs) ᐀1(e1)᐀1(e2) = 0, 2 3 3 3 2 (16) ᐀1( f1)᐀1( f2) − (r +s )᐀1( f2)᐀1( f1)᐀1( f2)+(rs) ᐀1( f2) ᐀1( f1) = 0.

Proof. By direct calculation, we have

1 (17) ᐀ (e )᐀ (e ) = − E (−ω0−1 f ) 1 2 1 1 s3(r + s)1 1112 1 1 1 = s3᐀ (e )᐀ (e ) − E . 1 1 1 2 rs2(r + s) 112

Hence, to prove (15) is equivalent to prove

3 ᐀1(e2)E112 − r E112᐀1(e2) = 0.

However, the latter is given by Lemma 3.7. The proof of (16) is analogous.

To prove that ᐀2 preserves the Serre relations, we also need auxiliary lemmas. Write −3 E21 := (adl e2)(e1) = e2e1 − r e1e2,

−3 and note that (G5)1 is equivalent to (adl e2)(E21) = e2 E21 − s E21e2 = 0, i.e., 3 E21e2 = s e2 E21.

3 − 2 + 3 2 − 2 3 3 = Lemma 3.9. e1 E21 s1E21e1 E21 rs 1E21e1 E21 (rs ) E21e1, f1 0.

[ ]= −3 = 2 [ 2 ]= −3 −1 + Proof. Since E21, f1 r 1e2ω1, ω1 E21 rs E21ω1, E21, f1 r s (r s) · 0 = 2 0 [ 3 ] = −3 −2 2 2 1E21e2ω1, ω1 E21 r sE21ω1, and E21, f1 r s 1 E21e2ω1, we get

ω −ω0 ω −ω0 ω −ω0 ω −ω0 6 = 1 1 E3 − (rs2)3 E3 1 1 − s1E 1 1 E2 + rs31E2 1 1 E 1 r−s 21 21 r−s 21 r−s 21 21 r−s 21 3 3 0 2 2 0 = −(rs) 1E21ω1 + rs 1E21ω1 E21 = 0 and THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 341 3 2 3 2 2 3 3 e1 E21 − s1E21e1 E21 + rs 1E21e1 E21 − (rs ) E21e1, f1 3 3 2 3 3 3 = e1[E21, f1] + [e1, f1]E21 − (rs ) E21[e1, f1] + [E21, f1]e1 2 2 2 − s1 E21e1[E21, f1] + E21[e1, f1]E21 + [E21, f1]e1 E21 3 2 2 2 + rs 1 E21e1[E21, f1] + E21[e1, f1]E21 + [E21, f1]e1 E21 ω −ω0 = r −3s−212e E2 e ω + 1 1 E3 1 21 2 1 r−s 21 ω −ω0 − s1 r −3s−1(r+s)1E e E e ω + E 1 1 E2 + r −31e ω e E2 21 1 21 2 1 21 r−s 21 2 1 1 21 ω −ω0 + rs31 r −31E2 e e ω + E2 1 1 E + r −3s−1(r+s)1E e ω e E 21 1 2 1 21 r−s 21 21 2 1 1 21 ω −ω0 − (rs2)3 E3 1 1 + r −3s−212 E2 e ω e 21 r−s 21 2 1 1 −3 −2 2 −3 −2 2 = 61 + (r s 1 ) 62 ω1 = (r s 1 ) 62 ω1, where

2 2 2 3 2 62 = e1 E21e2 − s (r + s)E21e1 E21e2 − (rs ) e2e1 E21 5 2 3 5 4 5 2 + rs E21e1e2 + r s (r + s)E21e2e1 E21 − r s E21e2e1.

3 −3 We next show 62 = 0. As E21e2 = s e2 E21 and e2e1 − r e1e2 = E21, we get

2 2 3 2 5 2 4 5 2 62 = e1 E21e2 − (rs ) e2e1 E21 + rs E21e1e2 − r s E21e2e1 2 3 5 − s (r+s)E21e1 E21e2 + r s (r+s)E21e2e1 E21 2 3 3 4 5 3 3 5 3 = −(rs ) E21 − r s E21 + r s (r + s)E21 = 0.

This completes the proof.

3 − 2 + 3 2 − 2 3 3 = Lemma 3.10. e1 E21 s1E21e1 E21 rs 1E21e1 E21 (rs ) E21e1, f2 0.

Proof. Noting that

−3 0 0 3 0 [E21, f2] = −r ω2e1, E21ω2 = r ω2 E21, 2 −3 0 3 [E21, f2] = −r ω2(e1 E21 + r E21e1), 3 −3 0 2 3 6 2 [E21, f2] = −r ω2(e1 E21 + r E21e1 E21 + r E21e1), 342 NAIHONG HU AND QIANSHI we obtain

3 2 3 2 2 3 3 e1 E21 − s1E21e1 E21 + rs 1E21e1 E21 − (rs ) E21e1, f2 3 2 2 = e1[E21, f2] − s1 E21e1[E21, f2] + [E21, f2]e1 E21 3 2 2 2 3 3 + rs 1 E21e1[E21, f2] + [E21, f2]e1 E21 − (rs ) [E21, f2]e1 −3 0 3 2 3 6 2 = −r ω2 s e1 e1 E21 + r E21e1 E21 + r E21e1 3 3 2 2 − s1 (rs) E21e1 e1 E21 + r E21e1 + e1 E21 3 2 3 2 2 3 + rs 1 (r s) E21e1 + (e1 E21 + r E21e1)e1 E21 2 3 2 3 6 2 − (rs ) (e1 E21 + r E21e1 E21 + r E21e1)e1 −2 0 = −r s ω2 S, where

2 3 3 2 2 2 2 2 2 S = (rs) (r −s ) e1 E21e1 + E21e1 E21 + s (2r +rs+s )(e1 E21) 5 3 2 2 2 2 2 6 2 2 − r s (2s +rs+r )(E21e1) − (r+s) e1 E21 − (rs) E21e1 . It remains to prove that S = 0, which by [Benkart et al. 2006, Lemma 3.4] is equivalent to showing that [S, f1] = 0 = [S, f2]. To this end, we ﬁrst observe: 3 − 2 + 3 2 − 2 3 3 = Lemma 3.11. e1 E21 s1e1 E21e1 rs 1e1 E21e1 (rs ) E21e1 0. Proof. It is easy to see that

3 2 3 2 2 3 3 −3 4 e1 E21 − s1e1 E21e1 + rs 1e1 E21e1 − (rs ) E21e1 = r (adl e1) (e2), −3 which is in fact the (r, s)-Serre relation (G5)2 up to a factor r .

Now set Si := [S, fi ]. Using the equations at the bottom of page 341, we obtain after some manipulations (see [Hu and Shi 2006] for details)

2 3 3 2 2 2 S2 = (rs) (r −s ) e1[E21, f2]e1 + [E21, f2]e1 E21 + E21e1[E21, f2] 2 2 2 + s (2r +rs+s ) e1[E21, f2]e1 E21 + e1 E21e1[E21, f2] 5 3 2 2 − r s (2s +rs+r ) [E21, f2]e1 E21e1 + E21e1[E21, f2]e1 2 2 6 2 2 − (r + s) e1[E21, f2] − (rs) [E21, f2]e1 −3 2 3 3 0 3 2 3 2 2 3 3 = −r (rs) (r + s )ω2 e1 E21 − s1e1 E21e1 + rs 1e1 E21e1 − (rs ) E21e1 , which vanishes by Lemma 3.11. Next we prove that S1 = 0. Using the formulas at the very beginning of the proof of Lemma 3.9 and noting that r + s sω − rω0 [e2, f ] = · 1 1 e , 1 1 rs r − s 1 THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 343 we can express S1 as the sum A + B + C + D, where

2 0 2 A := (rs) 1(ω1−ω1)E21e1 ω −ω0 (r+s)2 − r 5s3(2s2+rs+r 2)E 1 1 E e + (rs)5 E2 (sω −rω0 )e , 21 r−s 21 1 r−s 21 1 1 1 0 B := (rs)(r+s)1E21(sω1−rω1)e1 E21 ω −ω0 ω −ω0 + s2(2r 2+rs+s2) 1 1 E e E − r 5s3(2s2+rs+r 2)E e E 1 1 , r−s 21 1 21 21 1 21 r−s 2 2 0 C := (rs) 1e1 E21(ω1−ω1) ω −ω0 (r+s)2 sω −rω0 + s2(2r 2+rs+s2)e E 1 1 E − 1 1 e E2 , 1 21 r−s 21 rs r−s 1 21 1 D := (rs)2(r 3−s3)s−1(r+s)e E e ω e + e ω e2 E + E e2e ω r 3 1 21 2 1 1 2 1 1 21 21 1 2 1 2 2 2 + s (2r +rs+s ) e1e2ω1e1 E21 + e1 E21e1e2ω1 5 3 2 2 − r s (2s +rs+r ) e2ω1e1 E21e1 + E21e1e2ω1e1 2 −1 2 6 2 − (r+s) s e1 E21e2ω1 − (rs) E21e2ω1e1 .

= 2 0 = 2 0 Noting that ω1 E21 rs E21ω1 and ω1 E21 r sE21ω1, we obtain the simpliﬁed expressions 5 4 3 3 2 A = −r s (r −s )E21e1ω1, B = 0, 2 3 3 2 C = rs (r −s )e1 E21ω1.

3 −3 For the last summand, a calculation using the equalities E21e2 =s e2 E21, r e1e2 = −3 e2e1 − E21 and e2e1 = E21 + r e1e2 leads to

3 3 5 4 2 2 2 D = (r −s ) r s E21e1 − rs e1 E21 ω1

(see [Hu and Shi 2006] for details), showing that S1 = A + B + C + D = 0. This completes the proof of Lemma 3.10.

The next identity is a consequence of Lemmas 3.9, 3.10 and [Benkart et al. 2006, Lemma 3.4].

3 − 2 + 3 2 − 2 3 3 = Lemma 3.12. e1 E21 s1E21e1 E21 rs 1E21e1 E21 (rs ) E21e1 0.

Lemma 3.13. ᐀2 preserves the (r, s)-Serre relations (G5)2, (G6)2 into its associated object Us−1,r −1 (G2). 344 NAIHONG HU AND QIANSHI

Proof. For the fourth-degree (r, s)-Serre relation (G5)2, we have to prove that 6 4 3 2 2 3 (rs) ᐀2(e1) ᐀2(e2) − (rs) (r+s)(r +s )᐀2(e1) ᐀2(e2)᐀2(e1) 2 2 2 2 2 2 + (rs)(r +s )(r +rs+s )᐀2(e1) ᐀2(e2)᐀2(e1) 2 2 3 4 − (r+s)(r +s )᐀2(e1)᐀2(e2)᐀2(e1) + ᐀2(e2)᐀2(e1) vanishes. By virtue of the commutation relation in (13), this is equivalent to

3 2 3 2 2 3 3 e1᐀2(e1) − s1᐀2(e1)e1᐀2(e1) + rs 1᐀2(e1) e1᐀2(e1) − (rs ) ᐀2(e1) e1 = 0.

3 3 However, since ᐀2(e1) = e1e2 − r e2e1 = (−r )E21, the above identity is exactly the one given by Lemma 3.12. Similarly, we can verify that ᐀2 preserves the (r, s)-Serre relation (G6)2 into its associated object Us−1,r −1 (G2).

Lemma 3.14. ᐀1 preserves the (r, s)-Serre relations (G5)2, (G6)2 into its associated object Us−1,r −1 (G2).

Proof. For the fourth-degree (r, s)-Serre relation (G5)2, we have to prove that 6 4 3 2 2 3 (rs) ᐀1(e1) ᐀1(e2) − (rs) (r+s)(r +s )᐀1(e1) ᐀1(e2)᐀1(e1) 2 2 2 2 2 2 + (rs)(r +s )(r +rs+s )᐀1(e1) ᐀1(e2)᐀1(e1) 2 2 3 4 − (r+s)(r +s )᐀1(e1)᐀1(e2)᐀1(e1) + ᐀1(e2)᐀1(e1) = 0. In view of the commutation relation in (17), this is equivalent to

3 2 3 2 E112᐀1(e1) − r1᐀1(e1)E112᐀1(e1) + r s1᐀1(e1) E112᐀1(e1) 2 3 3 − (r s) ᐀1(e1) E112 = 0. We can further reduce this condition to

2 2 5 2 (18) E12᐀1(e1) − r (r + s)᐀1(e1)E12᐀1(e1) + r s᐀1(e1) E12 = 0, as a consequence of the commutative relation

2 −1 2 E112᐀1(e1) = rs ᐀1(e1)E112 + r s(r + s) E12, itself arising from the equalities

2 0 0 2 0 [E112 f1] = −(r + s) E12ω1, ω1 E112 = rs E112ω1. [ ] = − 0 Again, since E12, f1 1e2ω1, we have 2 −1 E12᐀1(e1) = r s᐀1(e1)E12 + r s1e2, = 3 =− 0−1 by which (18) is ﬁnally reduced to e2᐀1(e1) r ᐀1(e1)e2, since ᐀1(e1) ω1 f1. The proof of the second part is similar. THETWO-PARAMETERQUANTUMGROUPOFEXCEPTIONALTYPE G2 345

Theorem 3.15. ᐀1 and ᐀2 are the Lusztig symmetries from Ur,s(G2) to its associated quantum group Us−1,r −1 (G2) as ޑ-isomorphisms, inducing the usual Lusztig symmetries as ޑ(q)-automorphisms not only on the quantum group Uq (G2) of c Drinfel’d–Jimbo type but also on the centralized quantum group Uq (G2), only −1 when r = q = s .

References

[Benkart et al. 2006] G. Benkart, S.-J. Kang, and K.-H. Lee, “On the centre of two-parameter quantum groups”, Proc. Roy. Soc. Edinburgh Sect. A 136:3 (2006), 445–472. MR 2007a:17019 [Bergeron et al. 2006] N. Bergeron, Y. Gao, and N. Hu, “Drinfel’d doubles and Lusztig’s symmetries of two-parameter quantum groups”, J. Algebra 301:1 (2006), 378–405. MR MR2230338

[Hu and Shi 2006] N. Hu and Q. Shi, “Two-parameter quantum group of exceptional type G2 and Lusztig’s symmetries”, Preprint, 2006. math.QA/0601444 [Klimyk and Schmüdgen 1997] A. Klimyk and K. Schmüdgen, Quantum groups and their representations, Springer, Berlin, 1997. MR 99f:17017

Received August 16, 2005.

NAIHONG HU DEPARTMENT OF MATHEMATICS EAST CHINA NORMAL UNIVERSITY ZHONGSHAN NORTH ROAD 3663 SHANGHAI 200062 CHINA [email protected]

QIAN SHI DEPARTMENT OF MATHEMATICS EAST CHINA NORMAL UNIVERSITY ZHONGSHAN NORTH ROAD 3663 SHANGHAI 200062 CHINA PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

CONVERGENCE TO STEADY STATES FOR A ONE-DIMENSIONAL VISCOUS HAMILTON–JACOBI EQUATION WITH DIRICHLET BOUNDARY CONDITIONS

PHILIPPE LAURENÇOT

We investigate the convergence to steady states of the solutions to the one- 2 p dimensional viscous Hamilton–Jacobi equation ∂t u − ∂x u = |∂x u| , where (t, x) ∈ (0, ∞)×(−1, 1) and p ∈ (0, 1), with homogeneous Dirichlet boundary conditions. For that purpose, a Liapunov functional is constructed by the approach of Zelenyak (1968). Instantaneous extinction of ∂x u on a subinterval of (−1, 1) is shown for suitable initial data.

1. Introduction

Nonnegative solutions to the one-dimensional viscous Hamilton–Jacobi equation

2 p (1) ∂t u − ∂x u = a |∂x u| ,(t, x) ∈ (0, ∞) × (−1, 1), (2) u(t, ±1) = 0, t ∈ (0, ∞),

(3) u(0) = u0 ≥ 0, x ∈ (−1, 1), exhibit a rich variety of qualitative behaviours, according to the sign of a ∈ {−1, 1} and the values of p ∈ (0, ∞). On the one hand, extinction in ﬁnite time (that is, there is T? > 0 such that u(t) ≡ 0 for t ≥ T?) occurs for a = −1 and p ∈ (0, 1), while u(t) converges exponentially fast to zero as t → ∞ if a = −1 and p ≥ 1 [Benachour et al. 2007]. On the other hand, if a = 1 and p > 2, ﬁnite time gradient blow-up takes place for suitably large initial data [Souplet 2002] while convergence to zero of u(t) as t → ∞ still holds true for global solutions [Arrieta et al. 2004; Souplet and Zhang 2006]. In addition, all solutions are global for a = 1 and p ∈ [1, 2] and converge to zero as t → ∞ [Benachour et al. 2007; Souplet and Zhang 2006]. The case a = 1 and p ∈ (0, 1) offers an interesting novelty and is the subject of the present paper. Indeed, in contrast to the previous cases, the initial-boundary value problem (1)–(3) has a one parameter family (Uϑ )ϑ∈[0,1] of steady states when a = 1 and p ∈ (0, 1) with U1 ≡ 0 and Uϑ is not constant if ϑ ∈ [0, 1). These steady

MSC2000: primary 35B40; secondary 35K55, 37B25. Keywords: diffusive Hamilton–Jacobi equation, convergence to steady states, gradient extinction, Liapunov functional.

347 348 PHILIPPE LAURENÇOT states play an important role in the dynamics of solutions to (1)–(3): indeed, we will prove that any solution u to (1)–(3) converges as t → ∞ towards a steady state, which is nontrivial if, for instance, the initial datum u0 is nonnegative with a positive maximum. An interesting feature of Uϑ for ϑ ∈ (0, 1) is that they are constant on a subinterval of (−1, 1). This property is of course related to the fact that p ranges in (0, 1) and is reminiscent of the ﬁnite time extinction phenomenon already alluded to for nonnegative solutions when a = −1 and p ∈ (0, 1). It is p then natural to wonder whether the nonlinear term |∂x u| may induce a similar singular behaviour on the dynamics of u. More precisely, for a particular class of nonnegative initial data, we will show that the gradient ∂x u vanishes identically on [T?, ∞) × I for some T? > 0 and some subinterval I of (−1, 1). Let us point out here that, for nonnegative initial data, extinction in ﬁnite time cannot occur when a = 1 and p ∈ (0, 1), for the comparison principle warrants that u is bounded from below by the solution to the linear heat equation with the same initial and boundary data. From now on, we thus assume that

(4) a = 1 and p ∈ (0, 1), and

1 (5) u0 ∈ Y := w ∈ Ꮿ ([−1, 1]), w(±1) = 0 .

It then follows from [Benachour and Dabuleanu 2003, Theorem 3.1 and Propo- sition 4.1] that the initial-boundary value problem (1)–(3) has a unique classical solution u ∈ Ꮿ([0, ∞) × [−1, 1]) ∩ Ꮿ2,1((0, ∞) × (−1, 1)) satisfying

(6) min u0 ≤ u(t, x) ≤ max u0,(t, x) ∈ [0, ∞) × [−1, 1]. [−1,1] [−1,1] In addition, setting

(7) M(t) := max u(t, x), x∈[−1,1] the comparison principle ensures that t 7→ M(t) is a nonincreasing function of time and we put (8) M∞ := lim M(t) ∈ min u0, max u0 . t→∞ [−1,1] [−1,1] We recall that classical solutions to (1)–(3) enjoy the comparison principle; this may be proved by standard arguments, as in [Gilding et al. 2003, Theorem 4]. CONVERGENCE TO STEADY STATES FOR A HAMILTON–JACOBI EQUATION 349

Remark 1. The initial-boundary value problem (1)–(3) is actually well-posed in a larger space than Y , which depends on p, and we refer to [Benachour and Dab- uleanu 2003] for a more detailed account. Still, the solutions constructed in that reference belong to Y for any positive time. Since we are interested here in the large time behaviour, the assumption (5) that u0 ∈ Y is thus not restrictive. For further use, we also introduce the following notations: 2 − p (1 − p)α (9) α := and ᏹ := . 1 − p 0 2 − p We may now state our main result.

Theorem 2. Consider u0 ∈ Y and denote by u the corresponding classical solution to (1)–(3). Then M∞ ∈ [0, ᏹ0] and there is a nonnegative stationary solution us to (1)–(2) such that

(10) lim ku(t) − usk∞ = 0. t→∞

Furthermore, us 6≡ 0 and M∞ > 0 if Z 1 πx (11) u0(x) cos dx > 0. −1 2 It readily follows from the second assertion of Theorem 2 that the set of nontrivial and nonnegative steady states to (1)–(2) attracts all solutions to (1)–(3) starting from a nonnegative initial datum u0 6≡ 0. Observe however that the set of nontrivial and nonnegative steady states to (1)–(2) also attracts sign-changing solutions u to (1)–(3) since there are sign-changing initial data fulﬁlling (11).

The proof of Theorem 2 requires several steps and is performed as follows: we ﬁrst identify the stationary solutions to (1)–(2) in Section 2 and use them together with comparison arguments to establish that, if u0 ∈ Y is nonnegative with u0 6≡ 0, 1 then M∞ > 0 and {u(t); t ≥ 0} is bounded in Ꮿ ([−1, 1]) (Section 3). In Section 4, we employ the technique of [Zelenyak 1968] to construct a Liapunov functional for nonnegative solutions to (1)–(3). Let us mention here that this technique has also been used recently for related problems in [Arrieta et al. 2004; Simondon and Toure´ 1996]. For nonnegative initial data convergence towards a steady state then follows from the results of Section 3 and Section 4 by a LaSalle invariance principle argument. The large time behaviour of sign-changing initial data is next deduced from that of nonnegative solutions after observing that the negative part of any solution to (1)–(3) vanishes in a ﬁnite time (Section 6). Remark 3. A further outcome of Theorem 2 is that the large behaviour of solutions to (1) on a bounded interval is more complex for homogeneous Dirichlet boundary 350 PHILIPPE LAURENÇOT conditions than for periodic and homogeneous Neumann boundary conditions. In- deed, for the latter boundary conditions, it follows from [Benachour and Dabuleanu 2005; Benachour et al. 2002] that there are T? > 0 and m? ∈ ޒ such that u(t) ≡ m? for t ≥ T? whatever the signs of a and u0 are.

In Section 7, we prove the extinction in ﬁnite time of ∂x u on a subinterval of (−1, 1) for a speciﬁc class of initial data. More precisely, we have the following result:

Theorem 4. Assume further that there are m0 ∈ (0, ᏹ0) and ε > 0 such that

α 1+α (12) m0 − ᏹ0 |x| + ε |x| ≤ u0(x) ≤ m0, x ∈ [−1, 1].

Then, for each t ∈ (0, ∞), there is X(t) ∈ (0, 1) such that

u(t, x) = m0 for x ∈ (−X (t), X (t)).

Furthermore, if

1/α m0 (13) δ0 := 1 − ∈ (0, 1), ᏹ0 and δ ∈ (0, δ0), there exists T (δ) > 0 such that

u(t, x) = m0 for (t, x) ∈ [T (δ), ∞) × [−δ, δ].

An example of initial datum in Y fulﬁlling (12) is the following: u0(x) = ᏹ0 − α β ε − ᏹ0 |x| + ε |x| for x ∈ [−1, 1], where β ∈ (α, α + 1] and ε ∈ (0, αᏹ0/β).

The second assertion of Theorem 4 shows that ∂x u vanishes identically after some time on a subinterval of [−1, 1], a phenomenon which one could call ﬁnite time incomplete extinction in comparison to what occurs for periodic or homogeneous Neumann boundary conditions. But the ﬁrst assertion of Theorem 4 reveals that the extinction mechanism is somewhat stronger since, even if ∂x u0(x) vanishes only for x = 0, ∂x u vanishes instantaneously on a subinterval of [−1, 1] with positive measure.

Another consequence of Theorem 4 and (6) is that ku(t)k∞ = m0 for every t ≥ 0. Therefore, for an initial datum u0 in Y satisfying (12), the corresponding solution u to (1)–(3) does not obey the strong maximum principle.

The proof of Theorem 4 relies on comparison arguments with travelling wave solutions to (1) and is similar to that of [Gilding 2005, Theorem 9], some care being needed to cope with the boundary conditions. CONVERGENCE TO STEADY STATES FOR A HAMILTON–JACOBI EQUATION 351

Notations. Throughout the paper, we denote by r+ := max {r, 0} the positive part of the real number r. For r ∈ ޒ and s ∈ ޒ, we put r ∨ s := max {r, s} and r ∧ s := q min {r, s}. Also, for q ∈ [1, ∞], k.kq denotes the L (−1, 1)-norm.

2. Nonnegative steady states

In this section, we look for nonnegative stationary solutions to (1), (2), that is, nonnegative functions U ∈ Ꮿ2([−1, 1]) such that 2 p d U dU (14) + = 0, x ∈ (−1, 1), dx2 dx (15) U(±1) = 0.

Proposition 5. Let U ∈ Ꮿ2([−1, 1]) be a nonnegative solution to (14), (15). Then there is ϑ ∈ [0, 1] such that U = Uϑ , where α α Uϑ (x) := ᏹ0 (1 − ϑ) − (|x| − ϑ)+ , x ∈ [−1, 1].

Observe that Uϑ is constant on [−ϑ, ϑ] for each ϑ ∈ (0, 1) and that U1 ≡ 0. Proof. Let U ∈ Ꮿ2([−1, 1]) be a nonnegative solution to (14), (15). Then U is concave by (14) and we infer from the nonnegativity of U and the boundary conditions (15) that dU/dx(−1) ≥ 0 and dU/dx(1) ≤ 0. If dU/dx(−1) = 0, the concavity of U entails that U is a nonincreasing function in (−1, 1). Consequently, U ≡ 0 = U1 to comply with the boundary conditions (15). Similarly, if dU/dx(1) = 0, it follows from the concavity of U that U is non- decreasing on (−1, 1), whence U ≡ 0 = U1 by (15). We ﬁnally consider the case where dU/dx(−1) > 0 and dU/dx(1) < 0 and put

xI := sup {X ∈ (−1, 1) such that dU/dx(x) > 0 on [−1, X)},

xS := inf {X ∈ (−1, 1) such that dU/dx(x) < 0 on (X, 1]}.

Owing to the continuity of dU/dx, we have −1 < xI ≤ xS < 1 and dU/dx(x) = 0 for x ∈ [xI , xS] by the concavity of U. Direct integration of (14) then entails that there are two constants A and B such that −p dU dU A if x ∈ (xS, 1], (16) (x) (x) + (1 − p) x = dx dx B if x ∈ [−1, xI ).

Since p ∈ (0, 1) and dU/dx vanishes for x ∈ {xI , xS}, we may let x → xI and x → xS in (16) to deduce that A = (1 − p) xS and B = (1 − p) xI . We next integrate (16) to obtain that there are two constants CI and CS such that α CS − ᏹ0 (x − xS) if x ∈ (xS, 1], U(x) = α CI − ᏹ0 (xI − x) if x ∈ [−1, xI ). 352 PHILIPPE LAURENÇOT

Requiring the boundary conditions (15) to be fulﬁlled provides the values of CI and CS, whence

α α ᏹ0 (1 − xS) − ᏹ0 (x − xS) if x ∈ (xS, 1], U(x) = α α ᏹ0 (xI + 1) − ᏹ0 (xI − x) if x ∈ [−1, xI ).

Now, since dU/dx vanishes for x ∈ [xI , xS], we shall have U(xS) = U(xI ), which implies that 1 − xS = xI + 1, whence xS = −xI . Thus, necessarily, xS ∈ [0, 1], = from which the equality U UxS readily follows.

It is worth mentioning that kUϑ k∞ ≤ ᏹ0 for each ϑ ∈ [0, 1]. Combining this property with the convergence to a steady state to be proved in Section 5, we will conclude that M∞ ≤ ᏹ0. Remark 6. Proposition 5 shows in particular that there is nonuniqueness of classical solutions to (14), (15). A similar construction is performed in [Alaa and Pierre 1993; Lions 1985] for the boundary-value problem

−1u = |∇u|p in B(0, 1), u = 0 on ∂ B(0, 1), where B(0, 1) denotes the open unit ball of ޒN , N > 1, to establish the nonuniqueness of weak solutions for p > N/(N − 1).

3. Some properties of {u(t) ; t ≥ 0}

Introducing the positive cone Y+ := {w ∈ Y such that w ≥ 0} of Y , we ﬁrst prove that M∞ > 0 for u0 ∈ Y+, u0 6≡ 0, by constructing suitable subsolutions to (1)–(3) with the help of U0.

Lemma 7. Let u0 ∈ Y+ and denote by u the corresponding classical solution to (1)–(3). If u0 6≡ 0, we have M∞ > 0.

Proof. Since u0 6≡ 0, there are x0 ∈ (−1, 1), δ ∈ (0, 1) and m > 0 such that (x0 − δ, x0 + δ) ⊂ (−1, 1) and

(17) u0(x) ≥ m for x ∈ (x0 − δ, x0 + δ).

We put x1 := (x0 − 1) ∨ (−1), x2 := (x0 + 1) ∧ 1, J := [x1, x2], m λ := 1 ∧ , ᏹ0 − U0(δ) and v(x) := λ (U0(x − x0) − U0(δ)) for x ∈ J. On the one hand, it follows from (1) and (14) that

2 p p p 2 p ∂t v − ∂x v − |∂x v| = (λ − λ ) |∂xU0(. − x0)| ≤ 0 = ∂t u − ∂x u − |∂x u| CONVERGENCE TO STEADY STATES FOR A HAMILTON–JACOBI EQUATION 353 on [0, ∞) × J. On the other hand, the nonnegativity of u0 and the maximum principle entail the nonnegativity of u which then warrants that

v(x1) ≤ v(x0 − δ) = 0 ≤ u(t, x1),

v(x2) ≤ v(x0 + δ) = 0 ≤ u(t, x2), while the choice of λ entails that

v(x) ≤ λ (ᏹ0 − U0(δ)) ≤ m ≤ u0(x) for x ∈ (x0 − δ, x0 + δ),

v(x) ≤ v(x0 ± δ) = 0 ≤ u0(x) for x ∈ J \ (x0 − δ, x0 + δ). We then infer from the comparison principle that u(t, x)≥v(x) for (t, x)∈[0, ∞)× J. In particular, M(t) = ku(t)k∞ ≥ u(t, x0) ≥ v(x0) = λ (ᏹ0 − U0(δ)) for each t ≥ 0, whence M∞ ≥ λ (ᏹ0 − U0(δ)) > 0. We now turn to the question of global boundedness of the trajectory {u(t) ; t ≥0} in Ꮿ1([−1, 1]).

Lemma 8. Let u0 ∈ Y+ and denote by u the corresponding classical solution to (1)–(3). There is a constant 3 > 0 depending only on ku0kW 1,∞(−1,1) and p such that

(18) ku(t)kW 1,∞(−1,1) ≤ 3 for t ≥ 0. Proof. We ﬁrst recall that {u(t) ; t ≥ 0} is bounded in L∞(−1, 1) by (6) and we are ∞ left with the proof that {∂x u(t) ; t ≥ 0} is bounded in L (−1, 1). For that purpose, we choose λ > 1 such that " 1/(1−p) # 2 ku k∞ ≥ k u k ∨ 0 (19) λ ∂x 0 ∞ −α . 1 − p (1 − 2 ) ᏹ0

Putting v := λU0, we ﬁrst notice that the condition λ > 1 ensures that 2 p p p ∂t v − ∂x v − |∂x v| = (λ − λ ) |∂xU0| ≥ 0 in (0, ∞) × (−1, 1), while v(±1) = u(t, ±1) = 0 for each t ≥ 0. Next, on the one hand, it follows from (19) and the monotonicity properties of U0 that, if x ∈ (−1/2, 1/2), we have −α v(x) = λ U0(x) ≥ λ U0(1/2) = λ ᏹ0 (1 − 2 ) ≥ ku0k∞ ≥ u0(x). On the other hand, if x ∈ [1/2, 1], we have by (19) that Z 1 Z 1 dU0 1/(1−p) v(x) = λ (U0(x) − U0(1)) = λ (y) dy = α λ ᏹ0 y dy x dx x Z 1 Z 1 Z 1 −1/(1−p) ≥ α λ ᏹ0 2 dy ≥ k∂x u0k∞ dy ≥ |∂x u0(y)|dy x x x

≥ u0(x). 354 PHILIPPE LAURENÇOT

A similar computation shows that v(x) ≥ u0(x) also holds true for x ∈ [−1, −1/2]. Therefore, v ≥ u0 in [−1, 1] and the previous analysis allows us to apply the comparison principle and conclude that u(t, x) ≤ v(x) for (t, x) ∈ [0, ∞)×[−1, 1]. In particular, if t ≥ 0 and x ∈ (0, 1), we have

u(t, x) − u(t, 1) u(t, x) v(x) v(x) − v(1) = ≥ = . x − 1 x − 1 x − 1 x − 1

1/(1−p) Letting x → 1, we deduce that ∂x u(t, 1) ≥ ∂x v(1) = −λ (1 − p) . Since u0 ≥ 0, the comparison principle ensures that u(t, x) ≥ 0 = u(t, 1) for x ∈ (0, 1), so that we also have ∂x u(t, 1) ≤ 0. Arguing in a similar way for x = −1, we end up with

1/(1−p) (20) |∂x u(t, ±1)| ≤ λ (1 − p) for t ≥ 0.

1/(1−p) We now put k := k∂x u0k∞ ∨ λ (1 − p) , z := ∂x u and ᏾ := {(t, x) ∈ (0, ∞) × (−1, 1), z(t, x) 6= 0}. In the neighbourhood of each point (t0, x0) of ᏾, p the function |∂x u| is smooth, and classical parabolic regularity theory implies that 1,2 z is Ꮿ in a neighbourhood of (t0, x0) and satisﬁes

2 p−2 ∂t z(t, x) − ∂x z(t, x) = p |z(t, x)| z(t, x) ∂x z(t, x).

Since {(t, x) ∈ (0, ∞) × (−1, 1), z(t, x) > k} ⊂ ᏾, we deduce from the previous identity and (20) that

1 1 d = Z k − k2 = − x 1 − | − |2 (z k)+ 2 (z k)+ ∂x z x=−1 ∂x (z k)+ dx 2 dt −1 x=1 p (z − k)+ + z − k |z|p p + 1 |z − k| x=−1 Z 1 2 = − |∂x (z − k)+| dx, −1 whence 2 2 k(z(t) − k)+k2 ≤ k(z(0) − k)+k2 = 0, the last equality being true thanks to the choice of k. Consequently, ∂x u(t, x) = z(t, x) ≤ k in [0, ∞) × [−1, 1]. By a similar argument, we also establish that ∂x u(t, x) = z(t, x) ≥ −k in [0, ∞) × [−1, 1]. Therefore,

1/(1−p) |∂x u(t, x)| ≤ k∂x u0k∞ ∨ λ (1 − p) for (t, x) ∈ [0, ∞) × [−1, 1], which completes the proof of Lemma 8. CONVERGENCE TO STEADY STATES FOR A HAMILTON–JACOBI EQUATION 355

4. A Liapunov functional

We now construct a Liapunov functional for nonnegative solutions to (1)–(3) with the help of the technique developed in [Zelenyak 1968]. Let u0 ∈ Y+ and denote by u the corresponding classical solution to (1)–(3) which is also nonnegative by the maximum principle. We look for a pair of functions 8 and % ≥ 0 such that

Z 1 Z 1 d 2 (21) 8 (u, ∂x u) dx = % (u, ∂x u) |∂t u| dx. dt −1 −1

Since ∂t u(t, ±1) = 0 by (2), the ﬁrst term of the right-hand side of this equality also reads

d Z 1 8 (u, ∂x u) dx dt −1 Z 1 = [∂18 (u, ∂x u) ∂t u + ∂28 (u, ∂x u) ∂x ∂t u] dx −1 Z 1 2 2 = ∂18 (u, ∂x u) − ∂1∂28 (u, ∂x u) ∂x u − ∂2 8 (u, ∂x u) ∂x u ∂t u dx, −1 and it is then natural to require that

2 2 ∂18 (u, ∂x u) − ∂1∂28 (u, ∂x u) ∂x u − ∂2 8 (u, ∂x u) ∂x u

= % (u, ∂x u) ∂t u p 2 = % (u, ∂x u) |∂x u| + ∂x u for (21) to hold true. Following [Zelenyak 1968], we realize that a sufﬁcient condition for the previous equality to be valid is

p (22) ∂18 (u, ∂x u) − ∂1∂28 (u, ∂x u) ∂x u = % (u, ∂x u) |∂x u| , 2 (23) −∂2 8 (u, ∂x u) = % (u, ∂x u) .

Performing the computations as in [Zelenyak 1968], we see that the functions

2−p |∂x u| −p 8 (u, ∂x u) := u − and % (u, ∂x u) := |∂x u| (2 − p)(1 − p) solve the differential system (22), (23). However, % is singular when ∂x u vanishes and it is not clear how to give a meaning to (21) for such a choice of functions 8 and %. Nevertherless, we have the following weaker result which turns out to be sufﬁcient for our purposes. 356 PHILIPPE LAURENÇOT

Proposition 9. For each t > 0 and δ ∈ (0, 1], we have

d Z 1 |∂ u(t, x)|2−p Z 1 |∂ u|2 (24) x − u(t, x) dx + t dx ≤ 0. − − 2 2p/2 dt −1 (2 p)(1 p) −1 |∂x u| + δ

Proof. We ﬁx δ ∈ (0, 1] and deﬁne ψε by

0 00 −p ψε(0) = ψε(0) = 0 and ψε (r) = (|r| ∨ ε) , r ∈ ޒ for ε ∈ (0, δ). We infer from (1) and (2) that

d Z 1 [ψε (∂x u) − u] dx dt −1 Z 1 0 = ψε (∂x u) ∂x ∂t u − ∂t u dx −1 Z 1 = 0 x=1 − 00 2 + ψε (∂x u) ∂t u x=−1 ψε (∂x u) ∂x u 1 ∂t u dx −1 Z 1 00 2 p = − ψε (∂x u) ∂x u + (|∂x u| ∨ ε) ∂t u dx −1 Z 1 00 p p = − ψε (∂x u) ∂t u + (|∂x u| ∨ ε) − |∂x u| ∂t u dx −1 Z 1 Z 1 |∂ u|p = − 00 u | u|2 dx − − x u dx ψε (∂x ) ∂t 1 p ∂t . −1 −1 ε +

On the one hand, since ε ∈ (0, δ), we have

2 21/2 |∂x u| ∨ ε ≤ |∂x u| + δ , so that Z 1 Z 1 2 00 2 |∂t u| ψε (∂x u) |∂t u| dx ≥ dx. 2 2p/2 −1 −1 |∂x u| + δ

On the other hand, introducing

 |r|pr  r − if |r| ≤ ε,  (p + 1)ε p ξ(r) :=  pε r  if |r| ≥ ε,  p + 1 |r| CONVERGENCE TO STEADY STATES FOR A HAMILTON–JACOBI EQUATION 357

0 p p we have ξ (r) = (1 − |r| /ε )+ and |ξ(r)| ≤ ε. Therefore, thanks to (1),

Z 1 |∂ u|p − x u dx 1 p ∂t −1 ε + Z 1 |∂ u|p Z 1 |∂ u|p ≤ − x 2u dx + p − x dx 1 p ∂x ε 1 p −1 ε + −1 ε + Z 1 p ≤ ∂x ξ (∂x u) dx + 2 ε −1 p p ≤ |ξ(∂x u(t, 1))| + |ξ(∂x u(t, −1))| + 2 ε ≤ 4ε .

Consequently, for each ε ∈ (0, δ), we have

Z 1 Z 1 2 d |∂t u| p (25) [ψε (∂x u) − u] dx + dx ≤ 4ε . 2 2p/2 dt −1 −1 |∂x u| + δ

It remains to pass to the limit in (25) as ε → 0. For that purpose, we notice that

−p 0 |r| r p 1−p ψ (r) − ≤ ε ε 1 − p 1 − p

2−p for r ∈ ޒ, so that (ψε) converges uniformly towards r 7→ |r| /((2− p)(1− p)) on ∞ compact subsets of ޒ. Recalling that ∂x u(t) belongs to L (−1, 1) by Lemma 8, we may let ε → 0 in (25) and obtain (24).

Remark 10. It turns out that, at least formally, the functional

Z 1 |∂ w(x)|2−p w 7→ x − w(x) dx −1 (2 − p)(1 − p) is also a Liapunov functional for (1)–(3) when p ∈ (1, 2), while

Z 1 w 7→ (|∂x w(x)| ln (|∂x w(x)|) − |∂x w(x)| − w(x)) dx −1 is a Liapunov functional for (1)–(3) when p = 1. For p > 2, (1)–(3) still have Liapunov functionals but of a different kind [Arrieta et al. 2004].

Corollary 11. We have

Z ∞ Z 1 2 (26) |∂t u(t, x)| dx dt < ∞. 0 −1 358 PHILIPPE LAURENÇOT

Proof. Let T > 0. We integrate (24) with δ = 1 over (0, T ) and use (18) and the nonnegativity of u to obtain Z T Z 1 |∂ u(t, x)|2 t dx dt p/2 0 −1 1 + 32 Z T Z 1 |∂ u(t, x)|2 ≤ t dx dt 2 p/2 0 −1 |∂x u(t, x)| + 1 Z 1 |∂ u(0, x)|2−p Z 1 |∂ u(T, x)|2−p ≤ x − u(0, x) dx − x − u(T, x) dx −1 (2−p)(1−p) −1 (2 − p)(1 − p) 2−p 1 2−p 2 k∂ u k∞ Z 2 k∂ u k∞ ≤ x 0 + u(T, x) dx ≤ x 0 + 2 3. (2 − p)(1 − p) −1 (2 − p)(1 − p) Since the right-hand side does not depend on T > 0, we deduce (26).

5. Convergence to steady states

Proof of Theorem 2: nonnegative initial data. Let u0 ∈ Y+, u0 6≡ 0, and denote by u the corresponding classical solution to (1)–(3). We consider an increasing sequence (tn)n≥1 of positive real numbers such that tn → ∞ as n → ∞ and deﬁne a sequence of functions (un)n≥1 by un(t, x) := u(tn + t, x) for (t, x) ∈ [0, 1] × [−1, 1] and n ≥ 1. We next denote by gn the solution to 2 (27) ∂t gn − ∂x gn = 0,(t, x) ∈ (0, 1) × (−1, 1),

(28) gn(t, ±1) = 0, t ∈ (0, 1),

(29) gn(0) = un(0) = u(tn), x ∈ (−1, 1), and put hn = un − gn. Then hn is a solution to 2 p (30) ∂t hn − ∂x hn = |∂x un| ,(t, x) ∈ (0, 1) × (−1, 1),

(31) hn(t, ±1) = 0, t ∈ (0, 1),

(32) hn(0) = 0, x ∈ (−1, 1). p q By Lemma 8, the sequence (|∂x un| ) is bounded in L ((0, 1) × (−1, 1)) for every q ∈ (1, ∞). Since hn is a solution to (30)–(32), we infer from [Ladyzenskajaˇ et al. q 2,q 1968, Theorem IV.9.1] that (hn) is bounded in {w ∈ L (0, 1; W (−1, 1)) , ∂t w ∈ Lq ((0, 1) × (−1, 1))} for every q ∈ (1, ∞). We may then use [Ladyzenskajaˇ et al. 1968, Lemma II.3.3] with q = 4 to deduce that there is β ∈ (0, 1) such that (hn) β/2,β and (∂x hn) are bounded in Ꮿ ([0, 1] × [−1, 1]). This last property together with the Arzela–Ascoli` theorem entail that (hn) and (∂x hn) are relatively compact in Ꮿ([0, 1] × [−1, 1]). CONVERGENCE TO STEADY STATES FOR A HAMILTON–JACOBI EQUATION 359

At the same time, it follows from Lemma 8 and classical regularity properties of the heat equation that (gn) is relatively compact in Ꮿ([0, 1]×[−1, 1]), while (∂x gn) is relatively compact in Ꮿ([τ, 1]×[−1, 1]) for each τ ∈ (0, 1). Consequently, there are a subsequence of (un) (not relabeled) and U ∈ Ꮿ([0, 1] × [−1, 1]) such that ∂xU ∈ Ꮿ((0, 1] × [−1, 1]) and

un −→ U in Ꮿ([0, 1] × [−1, 1]), (33) ∂x un −→ ∂xU in Ꮿ([τ, 1] × [−1, 1]) for every τ ∈ (0, 1). Now, since (un) satisﬁes (1), (2), a straightforward consequence of (33) is that

2 p 0 (34) ∂t U − ∂x U = |∂xU| in Ᏸ ((0, 1) × (−1, 1)). Furthermore, it follows from Corollary 11 that

Z 1 Z 1 Z 1+tn Z 1 2 2 lim |∂t un| dx dt = lim |∂t u| dx dt = 0. n→∞ n→∞ 0 −1 tn −1 By a weak lower semicontinuity argument, we infer from (33) and the previous identity that ∂t U = 0. Then U does not depend on time and thus belongs to 1 2 p Ꮿ ([−1, 1]). Furthermore, recalling (34), we conclude that ∂x U + |∂xU| = 0 in Ᏸ0(−1, 1). The already established regularity of U implies that U ∈ Ꮿ2([−1, 1]) and solves (14), (15). Consequently, by Proposition 5, there exists ϑ ∈ [0, 1] such that U = Uϑ and (un(0)) = (u(tn)) converges towards Uϑ in Ꮿ([−1, 1]) as n → ∞ by (33). In particular, recalling that M(t) is deﬁned by (7), we have

α ᏹ0 (1 − ϑ) = kUϑ k∞ = lim ku(tn)k∞ = lim M(tn) = M∞, n→∞ n→∞ whence M∞ ≤ ᏹ0 and 1/α M∞ (35) ϑ = 1 − . ᏹ0 Since this identity determines ϑ in a unique way, we deduce that the set of cluster points of {u(t) ; t ≥ 0} is reduced to a single point {Uϑ } with ϑ given by (35). The set {u(t) ; t ≥ 0} being relatively compact in Ꮿ([−1, 1]) by Lemma 8 and the Arzela–Ascoli` theorem, we ﬁnally conclude that ku(t) − Uϑ k∞ → 0 as t → ∞, whence (10). In addition, since u0 6≡ 0, Lemma 7 guarantees that ϑ < 1, so that Uϑ is indeed a nontrivial steady state to (1)–(3). We have thus proved that,

if u ∈ Y+, u 6≡ 0, then M∞ > 0 and there is ϑ ∈ [0, 1) such that (36) 0 0 ku(t) − Uϑ k∞ → 0 as t → ∞, and Theorem 2 holds true for nonnegative initial data. 360 PHILIPPE LAURENÇOT

6. Sign-changing solutions

We now show that the family (Uϑ )ϑ∈[0,1] of nonnegative steady states to (1)– (2) constructed in Proposition 5 also describes the large time behaviour of sign- changing solutions to (1)–(3). For that purpose, we ﬁrst establish that any solution to (1)–(3) becomes nonnegative after a ﬁnite time.

Lemma 12. Consider u0 ∈ Y and denote by u the corresponding classical solution to (1)–(3). Then there is T? > 0 such that u(t, x) ≥ 0 for (t, x) ∈ [T?, ∞)×[−1, 1]. Moreover, if u0 ≤ 0, then u(t, x) = 0 for (t, x) ∈ [T?, ∞) × [−1, 1].

Proof. We put u˜0(x) = 0 ∧ u0(x) for x ∈ [−1, 1] and u˜0(x) = 0 for x ∈ ޒ \ [−1, 1]. Since u˜0 is a nonpositive, bounded and continuous function in ޒ, we infer from [Gilding et al. 2003, Theorem 3] that there is a unique classical solution u˜ ∈ Ꮿ([0, ∞) × ޒ) ∩ Ꮿ1,2((0, ∞) × ޒ)) to the Cauchy problem

2 p (37) ∂t u˜ − ∂x u˜ = a |∂x u˜| ,(t, x) ∈ (0, ∞) × ޒ,

(38) u˜(0) =u ˜0, x ∈ ޒ. Furthermore, u˜ is nonpositive in (0, ∞) × ޒ and is thus clearly a subsolution to (1)–(3) since u˜0 ≤ u0. The comparison principle then entails that u˜(t, x) ≤ u(t, x) for (t, x) ∈ [0, ∞) × [−1, 1].

But, since u˜0 is a nonpositive, bounded and continuous function with compact support in ޒ, it follows from [Benachour et al. 2002; Gilding 2005] that u˜ enjoys the property of ﬁnite time extinction, that is, there is T? > 0 such that

u˜(t, x) = 0 for (t, x) ∈ [T?, ∞) × ޒ.

Combining these two facts yield the ﬁrst assertion of Lemma 12. Next, if u0 ≤ 0, we have also u ≤ 0 in [0, ∞) × [−1, 1] by (6) and u thus identically vanishes in [T?, ∞) × [−1, 1].

Proof of Theorem 2: sign-changing initial data. By Lemma 12, there is T? > 0 such that u(T?, x) ≥ 0 for x ∈ [−1, 1]. Then either u(T?) ≡ 0 and thus u(t) ≡ 0 for t ≥ T?, and u(t) converges towards U1 as t → ∞. Or u(T?) 6≡ 0 and we infer from (36) that there is ϑ ∈ [0, 1) such that u(t + T?) converges towards Uϑ as t → ∞, which completes the proof of the ﬁrst statement of Theorem 2. Assume next that u0 fulﬁls (11). Putting ϕ1(x) := cos (πx/2) for x ∈ [−1, 1] 2 2 2 and λ1 := π /4, we recall that −d ϕ1/dx = λ1ϕ1 in (−1, 1) with ϕ1(±1) = 0. p We infer from (1), (11) and the nonnegativity of ϕ1 and |∂x u| that Z 1 Z 1 −λ1t u(t, x) ϕ1(x) dx ≥ e u0(x) ϕ1(x) dx > 0 −1 −1 CONVERGENCE TO STEADY STATES FOR A HAMILTON–JACOBI EQUATION 361 for t ≥ 0. In particular, with the previous notations, we have u(T?) ≥ 0 with Z 1 u(T?, x) ϕ1(x) dx > 0, −1 which, together with the positivity of ϕ1 on (−1, 1), ensures that u(T?) is nonnegative with u(T?) 6≡ 0. Arguing as before, we infer from (36) that there is ϑ ∈ [0, 1) such that u(t) converges towards Uϑ as t → ∞, which completes the proof of the second statement of Theorem 2.

7. Partial extinction of ∂x u in ﬁnite time Before proceeding with the proof of Theorem 4, we recall that, if σ ∈ (0, ∞) and µ ∈ ޒ, the function (t, x) 7→ µ + Wσ (x − σt) is a travelling wave solution to 2 p ∂t w − ∂x w = |∂x w| in (0, ∞) × ޒ (see [Gilding and Kersner 2004, Chapter 13], for instance), where Z ξ −1/(1−p) −σ (1−p)η1/(1−p) (39) Wσ (ξ) := −σ 1 − e + dη, ξ ∈ ޒ. 0 α Introducing W0(ξ) = −ᏹ0 ξ+ for ξ ∈ ޒ, we claim that

1+α (40) 0 ≤ Wσ (ξ) − W0(ξ) ≤ σ κp ξ+ , ξ ∈ ޒ,

α −r 2 with κp := (1 − p) /(2(3 − 2p)). Indeed, introducing ζ(r) := (r − 1 + e )/r and ζ1(r) := rζ(r) for r ≥ 0, we have for ξ ≥ 0 Z ξ 1/(1−p) 1/(1−p) Wσ (ξ) − W0(ξ) = ((1 − p)η) 1 − (1 − ζ1(σ (1 − p)η)) dη. 0

We deduce from the elementary inequalities 0 ≤ ζ1(r) ≤ 1 for r ≥ 0 and r (1 − r)1/(1−p) ≥ 1 − , r ∈ [0, 1], 1 − p that Wσ (ξ) − W0(ξ) ≥ 0 and Z ξ 1/(1−p) ζ1(σ (1 − p)η) Wσ (ξ) − W0(ξ) ≤ ((1 − p)η) dη. 0 1 − p We next use the fact that ζ(r) ≤ 1/2 for r ≥ 0 to complete the proof of (40).

Proof of Theorem 4. As mentioned, the proof is similar to that of [Gilding 2005, Theorem 9], the main difference being due to the boundary conditions. We never- theless reproduce the whole argument here for the sake of completeness. We ﬁrst observe that (12) implies that u0(x) ≥ m0 − ᏹ0 + U0(x) for x ∈ [−1, 1] and that 362 PHILIPPE LAURENÇOT m0 − ᏹ0 + U0 is a subsolution to (1) with m0 − ᏹ0 + U0(±1) ≤ 0. We then infer from the comparison principle and (6) that

(41) m0 − ᏹ0 + U0(x) ≤ u(t, x) ≤ m0 for (t, x) ∈ [0, ∞) × [−1, 1]. In particular,

(42) u(t, 0) = m0 for t ∈ [0, ∞).

We now consider σ ∈ (0, ε/κp) and put wσ (t, x) = m0 +Wσ (x −σ t) for (t, x) ∈ [0, ∞) × ޒ (recall that ε and m0 are both deﬁned in (12)). We readily have that 2 p 2 p (43) ∂t wσ − ∂x wσ − |∂x wσ | = 0 = ∂t u − ∂x u − |∂x u| in (0, ∞) × (0, 1) with

(44) wσ (t, 0) = m0 = u(t, 0), t ≥ 0, by (39) and (42). In addition, we infer from (12), (40) and the choice of σ that, for x ∈ [0, 1],

(45) wσ (0, x) = m0 + Wσ (x) = m0 + W0(x) + Wσ (x) − W0(x) α 1+α α 1+α ≤ m0 − ᏹ0 x + σ κp x ≤ m0 − ᏹ0 x + ε x

≤ u0(x).

Finally, if δ ∈ (0, δ0) and t ∈ [0, δ/σ], it follows from (40) that

(46) wσ (t, 1) = m0 + Wσ (1 − σt)

= m0 + W0(1 − σt) + Wσ (1 − σ t) − W0(1 − σt) α 1+α ≤ m0 − ᏹ0 (1 − σ t) + σ κp (1 − σ t) α α ≤ ᏹ0 (1 − δ0) − (1 − δ) + σ κp ≤ 0 as soon as σ is sufﬁciently small. Owing to (43), (44), (45) and (46), there is σδ depending only on p, m0, ε and δ such that, if σ ∈ (0, σδ), we may apply the comparison principle on [0, δ/σ] × [0, 1] to deduce that

(47) wσ (t, x) ≤ u(t, x), (t, x) ∈ [0, δ/σ] × [0, 1].

Recalling (41), we conclude from (47) that, if σ ∈ (0, σδ),

(48) u(t, x) = m0 for t ∈ [0, δ/σ] and x ∈ [0, σt]. A ﬁrst consequence of (47) is that, if t > 0, we may ﬁnd σ small enough such that σ ∈ (0, σδ) and t ∈ [0, δ/σ]. It then follows from (48) that u(t, x) = m0 for x ∈ [0, X (t)] with X(t) := σ t. CONVERGENCE TO STEADY STATES FOR A HAMILTON–JACOBI EQUATION 363

As a second consequence of (47), we note that, if t ≥ T (δ) := δ/σδ, there is σ ∈ (0, σδ) such that t = δ/σ . Then u(t, x) = m0 for x ∈ [0, δ] by (48). To complete the proof of Theorem 4, it sufﬁces to notice that v : (t, x) 7→ u(t, −x) also solves (1)–(2) with initial datum x 7→ u0(−x) which satisﬁes (12). Then, v also enjoys the above two properties from which we deduce that we have also u(t, x) = m0 for x ∈ [−X (t), 0] for every t > 0 and u(t, x) = m0 for x ∈ [−δ, 0] and t ≥ T (δ), thus completing the proof of Theorem 4.

Acknowledgements

Part of this work was done while the author enjoyed the hospitality and support of the Helsinki University of Technology and the University of Helsinki, within the Finnish Mathematical Society Visitor Program in Mathematics 2005–2006 on Function Spaces and Differential Equations. I also thank Sa¨ıd Benachour, Brian Gilding, Michel Pierre and Philippe Souplet for helpful discussions and comments, and the referee for pertinent remarks.

References

[Alaa and Pierre 1993] N. E. Alaa and M. Pierre, “Weak solutions of some quasilinear elliptic equations with data measures”, SIAM J. Math. Anal. 24 (1993), 23–35. MR 94g:35091 Zbl 0809.35021 [Arrieta et al. 2004] J. M. Arrieta, A. Rodriguez-Bernal, and P. Souplet, “Boundedness of global solutions for nonlinear parabolic equations involving gradient blow-up phenomena”, Ann. Sc. Norm. Super. Pisa Cl. Sci. (5) 3:1 (2004), 1–15. MR 2005b:35122 Zbl 1072.35098 [Benachour and Dabuleanu 2003] S. Benachour and S. Dabuleanu, “The mixed Cauchy–Dirichlet problem for a viscous Hamilton–Jacobi equation”, Adv. Differential Equations 8:12 (2003), 1409– 1452. MR 2005e:35108 Zbl 1101.35043 [Benachour and Dabuleanu 2005] S. Benachour and S. Dabuleanu, “Large time behavior for a viscous Hamilton–Jacobi equation with Neumann boundary condition”, J. Differential Equations 216:1 (2005), 223–258. MR 2006d:35104 Zbl 1078.35055 [Benachour et al. 2002] S. Benachour, P. Laurençot, D. Schmitt, and P. Souplet, “Extinction and non- extinction for viscous Hamilton-Jacobi equations in ޒN ”, Asymptot. Anal. 31:3-4 (2002), 229–246. MR 2003i:35125 Zbl 1046.35053 [Benachour et al. 2007] S. Benachour, S. Dabuleanu-Hapca,˘ and P. Laurençot, “Decay estimates for a viscous Hamilton-Jacobi equation with homogeneous Dirichlet boundary conditions”, Asymptot. Anal. 51:3-4 (2007), 209–229. q [Gilding 2005] B. H. Gilding, “The Cauchy problem for ut = 1u + |∇u| , large-time behaviour”, J. Math. Pures Appl. (9) 84:6 (2005), 753–785. MR 2006a:35149 Zbl 02244568 [Gilding and Kersner 2004] B. H. Gilding and R. Kersner, Travelling waves in nonlinear diffusion- convection reaction, Prog. Nonlinear Diff. Eq. Appl. 60, Birkhäuser, Basel, 2004. MR 2005k:35002 Zbl 1073.35002

[Gilding et al. 2003] B. H. Gilding, M. Guedda, and R. Kersner, “The Cauchy problem for ut = 1u + |∇u|q ”, J. Math. Anal. Appl. 284:2 (2003), 733–755. MR 2005h:35165 Zbl 1041.35026 364 PHILIPPE LAURENÇOT

[Ladyženskaja et al. 1968] O. A. Ladyženskaja, V. A. Solonnikov, and N. N. Ural’ceva, Linear and quasi-linear equations of parabolic type, Transl. Math. Monogr. 23, Amer. Math. Soc., Providence, RI, 1968. MR 39 #3159a Zbl 0174.15403 [Lions 1985] P.-L. Lions, “Quelques remarques sur les problèmes elliptiques quasilinéaires du second ordre”, J. Analyse Math. 45 (1985), 234–254. MR 87f:35088 Zbl 0614.35034 [Simondon and Touré 1996] F. Simondon and H. Touré, “A Lyapunov functional and long-time behaviour for a degenerate parabolic problem”, Adv. Math. Sci. Appl. 6:1 (1996), 243–266. MR 97a:35129 Zbl 0860.35064 [Souplet 2002] P. Souplet, “Gradient blow-up for multidimensional nonlinear parabolic equations with general boundary conditions”, Diff. Integral Eq. 15:2 (2002), 237–256. MR 2002h:35128 Zbl 1015.35016 [Souplet and Zhang 2006] P. Souplet and Q. S. Zhang, “Global solutions of inhomogeneous Ham- ilton–Jacobi equations”, J. Analyse Math. 99 (2006), 355–396. MR 2279557 [Zelenyak 1968] T. I. Zelenyak, “Stabilization of solutions of boundary value problems for a second order parabolic equation with one space variable”, 4 (1968), 34–45. In Russian; translated in Differential Equations 4 (1968), 17–22. MR 36 #6806 Zbl 0162.15301

Received October 24, 2005.

PHILIPPE LAURENC¸ OT MATHEMATIQUES´ POUR L’INDUSTRIEETLA PHYSIQUE, CNRS UMR 5640 UNIVERSITE´ PAUL SABATIER –TOULOUSE 3 118 ROUTEDE NARBONNE F-31062 TOULOUSECEDEX 9 FRANCE [email protected] http://www.mip.ups-tlse.fr/~laurenco/ PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

UNIQUENESS RESULTS FOR CONSTANT MEAN CURVATURE GRAPHS

LAURENT MAZET

We give two uniqueness results for the Dirichlet problem associated to the constant mean curvature equation, involving mean curvature graphs over strips of R2. The proofs are based on height estimates and the study of the asymptotic behavior of solutions to the Dirichlet problem.

Introduction

Surfaces with constant mean curvature are of great interest in mathematics: they model soap ﬁlms, for example, and appear as interfaces in isoperimetric problems. One viewpoint in studying such surfaces is to consider them as graphs. Let be a domain of ޒ2. The graph of a function u over has constant mean curvature H > 0 if it satisﬁes the partial differential equation ∇u = (CMC) div p 2H. 1 + |∇u|2 Thanks to the work of J. Serrin [1970; 1969] and J. Spruck [1972/73], we can build a lot of constant mean curvature graphs over bounded domains of ޒ2. Over unbounded domains, the Dirichlet problem associated to (CMC) is more complicated. R. Finn [1965] asked whether the graph of a solution u of (CMC) over the strip ޒ × (−1/(2H), 1/(2H)) must be a regular cylinder of radius 1/(2H). P. Collin [1990] and A. N. Wang [1990] then built counterexamples. Other examples of solutions over strips were given by R. Lopez´ [2001; 2002]. Our key results in this paper, Theorems 10 and 12, say that solutions are unique under the conditions of either the Collin–Wang or the Lopez´ examples. These examples are of particular interest because they include unbounded boundary data; uniqueness is already known in the case of bounded boundary data, and also when the boundary data is small with respect to ln r, where r is the distance to the origin [Huang 1995]. Our proofs involve two major steps. First, if there are two solutions for the same boundary data, the difference between these solutions cannot stay bounded. This

MSC2000: 53A10. Keywords: constant mean curvature, uniqueness in Dirichlet problem.

365 366 LAURENT MAZET yields information on the asymptotic behavior of the boundary data. In the second step, we analyze the consequences of this behavior for the asymptotic behavior of a solution, using the notion of an arc of divergence. This idea is similar to the one used by Tam [1987a], who applied it to the related uniqueness question for capillary surface problems (where the desired solution u of (CMC) in must satisfy ∇u · = p ν cos γ on ∂, 1 + |∇u|2 rather than a Dirichlet boundary condition; here ν is the outward unit normal to ∂ and γ is the wetting angle). See also [Tam 1987b; Hwang 1995]. The uniqueness question has been studied for the minimal surface equation ∇u = (MSE) div p 0. 1 + |∇u|2 Nitsche [1965] proved that over a strip or angular sector {y > |x| cot α}, with 0 < α < π/2, the only solution of (MSE) vanishing on the boundary is u ≡ 0. Hence he conjectured the uniqueness of the solution to the Dirichlet problem for (MSE) in such domains. Collin [1990] gave a counterexample; thus, in view of our Theorems 10 and 12, the uniqueness problem for (MSE) on strip domains stands in contrast with the same problem for (CMC).

1. The existence results of Collin–Wang and López

In this section we recall two existence results on the Dirichlet problem for the constant mean curvature equation (CMC) on a strip = ޒ × (−l, l) of width 2l. It was proved in [Lopez´ 2001] that the width needs to be at most 1/H for there to be a solution. The ﬁrst result we quote concerns the limiting case 2l = 1/H. For f : ޒ → ޒ a continuous function, we deﬁne ϕ f on ∂ by ϕ f (x, ±l) = f (x). Theorem [Collin 1990]. Let f : ޒ → ޒ be a convex continuous function. There exists a solution u of (CMC) on = ޒ×(−1/(2H), 1/(2H)) agreeing with ϕ f on the boundary. (Wang [1990] proved this for the convex function x 7→ x2.)

The second result, by Lopez,´ deals with the case where 2l < 1/H. We say that a domain U ⊂ ޒ2 satisﬁes an exterior R-circle condition if for each point p ∈ ∂U there is a disk D of radius R such that D ∩ U = {p}. This says a circle of radius R can roll outside U along ∂U touching each point of ∂U. A continuous function f : ޒ → ޒ is said to satisfy a lower R-circle condition if the domain {(x, y) ∈ ޒ2 | y ≥ f (x)} satisﬁes an exterior R-circle condition. Thus UNIQUENESS RESULTS FOR CONSTANT MEAN CURVATURE GRAPHS 367 a circle of radius R can roll under the graph of f touching each point of the graph along its motion.

Theorem [Lopez´ 2002]. Let f : ޒ → ޒ be a continuous function satisfying a lower ∗ ρt -circle condition, where t ∈ ޒ+ and ρt is the maximal radius of the nodoid neck with minimal radius t (see below). There exists a solution u of (CMC) on the strip = ޒ × (−ht , ht ) agreeing with ϕ f on the boundary, where ht is the half-height of the same nodoid neck.

The authors of these theorems use Perron’s technique to build their solutions as the supremum of subsolutions. The difﬁculty is ﬁnding good barrier functions to ensure the boundary value. Theorems 10 and 12 below state that the solutions built by these authors are unique for the boundary data ϕ f .

The one-parameter family of nodoids. Constant mean curvature surfaces of revolution are of two types, each forming a one-parameter family. Unduloids are embedded surfaces: as the parameter changes, the unduloid family goes from the cylinder of radius 1/(2H) into a stack of tangent sphere of radius 1/H. By contrast, nodoids are not embedded; their interest lies in that each nodoid contains a piece that looks like a catenoidal neck with mean curvature vector pointing outward:

radius ρt (H) height 2ht (H)

We recall the construction of nodoids and ﬁx notations; see [Delaunay 1841; Eells 1987; Lopez´ 2002] for details. Take the surface of revolution parametrized by (r(u) cos θ, r(u) sin θ, u), arising from a positive smooth function r(u) deﬁned on an open interval I . The normal vector is 1 N(u, θ) = √ (cos θ, sin θ, −r 0). 1 + r 02 The surface has constant mean curvature H if 1 r 00 2H = − √ + . r 1 + r 02 (1 + r 02)3/2 368 LAURENT MAZET

Multiplying by rr 0 and integrating we see there exists c ∈ ޒ such that r (1) Hr 2 = −√ + c. 1 + r 02 Since Hr 2 is positive, c needs to be positive. Then there exist h, ρ and a solution r : [−h, h] → [0, ρ] to (1) such that r is even and the initial value r(0) = t > 0 is the minimum of r. Moreover, r(h) = ρ, r 0(h) = +∞, and Hρ2 = c. The associated surface is a nodoid. For u = 0, we have Ht2 + t = c, so √ −1 + 1 + 4Hc t = ; 2H that is, t an increasing function of c with t = 0 for c = 0 and limc→+∞ t = +∞. We will use t as the parameter for the family of nodoids. We have s Ht2 + t Z ρ H(ρ2 − x2) = = = = ρt ρt (H) , ht ht (H) p dx. H t x2 − H 2(ρ2 − x2)2 To summarize:

Proposition 1. There exists a one-parameter family of nodoids {ᏺt , t > 0} with constant mean curvature H given by the rotation of a curve γt around the z-axis, with the following properties:

(1) The curve γt is a graph on [ht , ht ] of an even function.

(2) The curve γt has horizontal tangents at ±ht . The surface ᏺt is included in the slab ᏿t : |z| ≤ ht and is tangent to it. (3) The mean curvature vector points out of the bounded domain determined by ᏺt in the slab ᏿t . 2 2 2 (4) The circle Ct of ᏺt with smallest radius is given by x + y = t , z = 0.

(5) The function ht is strictly increasing on t and 1 lim ht = 0, lim ht = . t→0 t→+∞ 2H

(6) The function ρt (H) is strictly increasing and 1 lim ρt (H) = 0, lim ρt (H) = +∞, lim ρt (H) − t = . t→0 t→+∞ t→+∞ 2H

The two limits of ht and ρt (H) as t → +∞ allow us to consider Collin’s result as a limiting case of Lopez’s´ theorem. Indeed, when R goes to +∞, the uniform R-circle condition for f becomes convexity, since the circle becomes a line. UNIQUENESS RESULTS FOR CONSTANT MEAN CURVATURE GRAPHS 369

2. The maximal and minimal solutions

Solutions of the constant mean curvature Dirichlet problem (CMC) are bounded above by those of the corresponding zero mean curvature problem: Lemma 2. Let f : ޒ → ޒ be a continuous function. On = ޒ×(−l, l), there exists a solution w of the minimal surface equation (MSE) with w|∂ = ϕ f . Moreover, w ≥ u for every solution u of (CMC) on agreeing with ϕ f on the boundary. Proof. By [Jenkins and Serrin 1966], if n is a large enough integer, there exist + − ± solutions wn and wn of (MSE) on (−n, n) × (−l, l) with wn = ϕ f on (−n, n) × ± {−l, l} and wn = ±∞ on {−n, n} × (−l, l). Fix such solutions for each n large. + − + By the maximum principle, for every n and m, we have wn ≥ wm , and (wn ) + is a decreasing sequence. Thus (wn ) converges to a solution w of (MSE) on agreeing with ϕ f on the boundary. Now consider a solution u of (CMC) on with ϕ f as boundary value. By the + maximum principle, wn ≥ u for every n. Taking the limit, we see that w ≥ u. This gives an upper bound for u without any hypothesis on the function f . To get a lower bound we do need such hypotheses. The function c deﬁned on = ޒ × (−1/(2H), 1/(2H)) by r 1 1 c(x, y) = − − y2 + (x − x ) tan θ + z cos θ 4H 2 0 0 is a solution of (CMC): its graph is the half-cylinder with the two straight lines of equation z = (x − x0) tan θ + z0 over ∂ as boundary. Lemma 3. Let f : ޒ → ޒ be a convex function and let u be a solution of (CMC) on = ޒ × (−1/(2H), 1/(2H)) agreeing with ϕ f on the boundary. Take x0 ∈ ޒ and let z = (x − x0) tan θ0 + f (x0) be a straight line lying below the graph of f (such a line exists by convexity). Let c denote the half-cylinder associated to this line. Then u ≥ c on .

Proof. Let h be the function defined on by h(x, y) = (x −x0) tan θ0 + f (x0). We have u ≥ h on the boundary. If the function f is affine, f (x) = (x − x0) tan θ0 + f (x0), Theorem 8 in [Mazet 2006a] states that c is the only constant mean curvature extension for ϕ f . Then u = c. If f is not affine, the set of θ such that there exists x1 ∈ ޒ with z =(x−x1) tan θ+ f (x1) lies below the graph of f is an interval I ⊂ ޒ. We assume that θ0 is in the interior of this interval. For θ0 an end point of this interval, the property is proved by continuity. Since θ0 is in the interior of I , there exist x1 < x0 < x2 and θ1 < θ0 < θ2 such that (x − x1) tan θ1 + f (x1) ≤ f and (x − x2) tan θ2 + f (x2) ≤ f . By Proposition 3 370 LAURENT MAZET in [Mazet 2006a], there exists K ∈ ޒ+ such that

u(x, y) ≥ (x − x1) tan θ1 + f (x1) − K,

u(x, y) ≥ (x − x2) tan θ2 + f (x2) − K.

Since θ1 < θ0 < θ2, these two equations imply that u(x, y) ≥ h(x, y) if |x| is big enough. We have h ≥ c on ; then u ≥ c on ∂ and outside a compact of . By the maximum principle, u ≥ c in . In the case of the Lopez´ solutions, we get the following lower bound.

Lemma 4. Let f : ޒ → ޒ a continuous function that satisﬁes a lower ρt -circle condition. Let x be in ޒ and let Ꮿ be a circle of radius ρt that established the uniform ρt -circle condition at the point (x, f (x)). Let u be a solution of (CMC) on = ޒ × (−ht , ht ) agreeing with ϕ f on the boundary. Then the graph of u lies above the nodoid ᏺt having a horizontal axis and bounded by the two parallel circles Ꮿ in the vertical plane y = −ht and y = ht .

Proof. Let ez denote the vertical unit vector (0, 0, 1). For s in ޒ, we translate by sez the nodoid ᏺt bounded by the two parallel circles Ꮿ. For s negative enough, ᏺt + sez lies below the graph of u. Let s grow until the first contact. The mean curvature of the graph is upward pointing and the mean curvature of ᏺt points outward. So by maximum principle, the first contact cannot be an interior point. Then, because of the hypothesis on f , the first contact is at s = 0 and the lemma is proved. The estimates in the preceding two lemmas have important consequences for uniqueness. To begin with, we derive from them a technical lemma. Lemma 5. Let f : ޒ → ޒ be a continuous function such that either = × − 1 1 (1) ޒ 2H , 2H and f is convex, or (2) = ޒ × (−ht , ht ) and f satisfies a lower ρt -circle condition.

Let Ᏸ denote the set of all solutions u of (CMC) on agreeing with ϕ f on the + − boundary. For any u1, u2 ∈ Ᏸ, there exist v and v in Ᏸ such that + − v ≥ max(u1, u2), v ≤ min(u1, u2). Proof. For n ∈ ގ, deﬁne

p 2 2 p 2 2 n = (x, y) ∈ | − n − 1/(2H) − y ≤ x ≤ n + 1/(2H) − y . The boundary of is composed of two segments and two circle-arcs of curvature 2H. Following Perron’s method (see [Courant and Hilbert 1962] or [Gilbarg and + − + Trudinger 1983]), we build solutions vn and vn of (CMC) on n, with vn = − max(u1, u2) and vn min(u1, u2) on the boundary. UNIQUENESS RESULTS FOR CONSTANT MEAN CURVATURE GRAPHS 371

+ To build vn , we consider subsolutions, of which max(u1, u2) is one. By the maximum principle, every subsolution is less than the solution w of (MSE) given + by Lemma 2. We can then define vn as the supremum over all subsolutions, and this function takes the right boundary values on the two segments because max(u1, u2) equals w on it. For the two arcs of circle, we use the barrier functions built in [Serrin 1970]. − Similarly, we define vn as the infimum of all supersolutions, which exist since min(u1, u2) is one. Again by the maximum principle, every supersolution satisfies the lower bound in Lemma 3 or 4. The half-circles and nodoids of these same − lemmas are used as barrier functions and give us the boundary value of vn on the two segments. For the two arcs of circle, we use Serrin’s arguments. + + On n, we have max(u1, u2) ≤ vn ≤ w; thus a subsequence converges to v + + − on and v ∈ Ᏸ. Clearly max(u1, u2) ≤ v . The sequence vn is bounded above by min(u1, u2) and satisfies the lower bounds of Lemmas 3 or 4. Therefore a sub- − sequence converges to v a solution of (CMC) agreeing with ϕ f on the boundary. − Moreover, min(u1, u2) ≥ v . Proposition 6. Let f : ޒ → ޒ be a continuous function such that either = × − 1 1 (1) ޒ 2H , 2H and f is convex, or (2) = ޒ × (−ht , ht ) and f satisfies a lower ρt -circle condition.

There exist two solutions umax and umin of (CMC) on agreeing with ϕ f on the boundary and such that every solution u of (CMC) on agreeing with ϕ f on the boundary satisﬁes

umin ≤ u ≤ umax.

Proof. Denote by Ᏸ the set of all solutions u of (CMC) on agreeing with ϕ f on the boundary; by the work of Collin and Lopez,´ Ᏸ is nonempty. Deﬁne umax and umin at p ∈ by

umax(p) = sup u(p), umin(p) = inf u(p). u∈Ᏸ u∈Ᏸ

By Lemma 2, umax is well deﬁned; Lemmas 3 and 4 ensure that umin > −∞. As in the classical Perron process, it can be proved that umax and umin are solutions of (CMC) on : the argument we need is that for every u1 and u2 in Ᏸ there exist u3 ∈ Ᏸ bounding max(u1, u2) from above and u4 ∈ Ᏸ bounding min(u1, u2) from below. These are given by Lemma 5. Using the solution w of (MSE) built in Lemma 2, the half-cylinders of Lemma 3 or the nodoids of Lemma 4 as barrier functions, we ﬁnally prove that umax and umin have ϕ f as boundary value. The construction also gives, for every u ∈ Ᏸ,

umin ≤ u ≤ umax. 372 LAURENT MAZET

An important fact is that, for every (x, y) ∈ , these solutions satisfy

(2) umax(x, y) = umax(x, −y), umin(x, y) = umin(x, −y), because the functions (x, y) 7→ umax(x, −y) and (x, y) 7→ umin(x, −y) lie in Ᏸ.

Upper bounds. We now look for explicit upper bounds for solutions of (CMC).

Proposition 7. Let f : ޒ → ޒ be a continuous function and take x0 ∈ ޒ. Assume that f is monotonic on [x0, +∞). Let u be a solution of (CMC) on = ޒ×(−a, a) agreeing with ϕ f on the boundary. Then, for x ≥ x0 + 1/H, we have 1 u(x, y) ≤ f (x) + . 2H

Proof. We only consider the case where f is increasing on [x0, +∞). Take a ≥ x0+ 1/H and denote by C(s) the horizontal cylinder of axis {x = a −1/(2H)}∩{z = s} and radius 1/(2H). For s large, C(s) lies above the graph of u. Let s decrease down to the value s0 where the first contact happens. By the maximum principle, this first contact point is on the boundary at a point of first coordinate a0 ∈ [a −1/(2H), a]. 0 We have f (a ) ≥ s0 − 1/(2H). Since C(s) lies above the graph of u for every s ≥ s0, we have u(a, y) ≤ s. Thus 0 0 u(a, y) ≤ s0 ≤ f (a )+1/(2H). Since a < a and f is increasing, we conclude that u(a, y) ≤ f (a) + 1/(2H). We say that a function f : ޒ → ޒ satisfies an upper R-circle condition at a ∈ ޒ if −f satisfies a lower R-circle condition there. Remark. For large s, the disk with center (a, s) and radius R is entirely contained in {(x, y) ∈ ޒ2 | y ≥ f (x)}. As s decreases and first makes contact with the graph of f , we obtain an upper R-circle condition at the abscissa(s) of the contact point(s). As a changes, we get all the abscissas where f satisfies an upper R-circle condition. Thus for every a ∈ ޒ there exists a0 ∈ [a−R, a+R] where f satisfies an upper R- circle condition. Proposition 8. Let f : ޒ → ޒ be a continuous function. Let u be a solution of (CMC) on = ޒ × (−l, l) agreeing with ϕ f on the boundary. Assume the f satisfies an upper 1/(2H)-circle condition at x0 ∈ ޒ. Then u(x0, y) ≤ f (x0) for every y ∈ [−l, l].

Proof. Let 0 be a circle realizing the upper 1/(2H)-circle condition at x0. Denote by C(s) the horizontal cylinder of axis {x = a} ∩ {z = b + s} and radius 1/(2H), where (a, b) is the center of 0. For big s the cylinder C(s) lies above the graph of u; as s decreases, the ﬁrst contact with the graph of f happens for s = 0, because = { } × [− ] of maximum principle, Then, on the segment Ix0 x0 l, l , u is bounded above by f (x0). UNIQUENESS RESULTS FOR CONSTANT MEAN CURVATURE GRAPHS 373

Let f : ޒ → ޒ be a continuous function that satisfies a lower R-circle condition. Let a ∈ ޒ denote a point where f satisfies an upper R0-circle condition. Since at a there are circles both above and below the graph of f , the graph has a tangent there. Thus either f 0(a) exists or f 0(a) = ±∞; either way, the derivative of f at a has a well defined sign. We have an analog of Rolle’s Theorem: Lemma 9. Let f : ޒ → ޒ be a continuous function that satisfies a lower R-circle condition. Let a 0 and f 0(b) < 0, there exists c ∈ [a, b] such that (1) f satisfies an upper R0-circle condition at c, and (2) f 0(c) = 0. √ Proof. Let g denote the function defined by g(x) = R0 − R02 − x2 on [−R0, R0]; its graph is a half-circle of radius R0. Since f satisfies an upper R0-circle condition at a and f 0(a) > 0, f is upper bounded by f (a) + g(x − a) on [a−R0, a]. In the same way, f is bounded above by f (b) + g(x − b) on [b, b+R0]. Let c ∈ [a, b] denote a point where f (c) = max[a,b] f . Then f (x) is bounded above by m(x) on [a−R0, b+R0], where m(x) is defined by

 f (c) + g(x−a) for x ∈ [a−R0, a],  m(x) = f (c) for x ∈ [a, b],   f (c) + g(x−b) for x ∈ [b, b+R0].

0 0 This implies that f satisﬁes an upper R -circle condition at c, so f (c) = 0.

3. The uniqueness of Collin and Wang’s solutions

Theorem 10. If f : ޒ → ޒ is a convex function, there is a unique solution of (CMC) on = ޒ × (−1/(2H), 1/(2H)) agreeing with ϕ f on the boundary. Proof. Existence is Collin’s theorem (page 366); we prove uniqueness. By Propo- sition 6, there are two solutions umin and umax of (CMC) on agreeing with ϕ f on the boundary and such that, for every solution u of the same Dirichlet problem, umin ≤ u ≤ umax. Thus our task is to show that umin = umax. Suppose otherwise; then umax − umin is unbounded on , by [Miklyukov 1979; Hwang 1988; Collin and Krust 1991]. By interchanging x and −x if needed, we can assume that

(3) lim maxI (umax − umin) = +∞, x→+∞ x where Ix = {x} × [−1/(2H), 1/(2H)]. 0 0 Since f is convex, f has a left derivative fl and a right derivative fr at every 0 point. These two functions increase and have the same limit at +∞. If lim+∞ fl = 374 LAURENT MAZET

0 lim+∞ fr < +∞, f is lipschitz continuous on ޒ+. Then (3) is in contradiction with [Mazet 2006a, Theorem 5]. Thus f must satisfy (4) lim f 0 = lim f 0 = +∞. +∞ l +∞ r

Asymptotic behavior of umin. To proceed we must recall from [Tam 1987a; 1987b; Mazet 2006b] the notion of an arc of divergence. Let (vn) be a sequence of solutions of (CMC) and let Nn denote the upward pointing normal to the graph of vn. 1 Assume that Nn(P) tends to a horizontal unit vector (ν, 0), with ν ∈ ޓ . Let C denote the arc of circle in the xy-plane with radius 1/(2H) such that P lies in C and 2Hν is the curvature vector of C at P. We call C an arc of divergence of the sequence (vn). We extend ν to C by letting 2Hν(Q) be the curvature vector of C at Q ∈ C. Then Nn(Q) converges to (ν(Q), 0) for every Q ∈ C. 2 For u a function on a domain of ޒ , deﬁne the differential 1-form ωu by u u = x − y ωu p dy p dx, 1 + |∇u|2 1 + |∇u|2 where ux and u y are the partial derivatives of u. When u is a solution of (CMC), ωu satisﬁes dωu = 2H dx ∧ dy; see [Spruck 1972/73]. It follows from the previous paragraph that, for every subarc C0 of the arc of divergence C, we have Z 0 lim ωu = `(C ), →+∞ n C0 where `(C0) the length of C0. We orient C0 so that ν points toward the left along C0. For a ∈ ޒ, denote by C+(a) the arc of circle {x ≥ a} ∩ {(x − a)2 + y2 = 1/(4H 2)}. Its endpoints are (a, ±1/(2H)) and it contains the point (a + 1/(2H), 0).

Lemma 11. There exists an increasing real sequence (xn) such that lim xn = +∞ + and C (0) is an arc of divergence of the sequence (un) of solutions of (CMC) on , where un is deﬁned by un(x, y) = umin(x+xn, y).

Proof. Let vn be deﬁned on by vn(x, y) = umin(x +n, y). The boundary value of = + [− +∞ vn is ϕ fn with fn(x) f (x n). Because of (4), fn is increasing on 1/H, ) for n large. Hence, by Proposition 7, vn(0, 0) ≤ fn(0) + 1/(2H). Now let θn ∈ 0 0 [0, π/2) be such that fnl (0) ≤ tan θn ≤ fnr (0). By Lemma 3 we have r 1 1 1 1 v , ≥ − + θ + f ( ). n 0 2 tan n n 0 H cos θn 4H H

Again because of (4), θn converges to → π/2. Hence 1 1 1 − ≥ − 1 − −−−−→ +∞ vn , 0 vn(0, 0) sin θn 2 . H H cos θn 2H n→+∞ UNIQUENESS RESULTS FOR CONSTANT MEAN CURVATURE GRAPHS 375

Thus the sequence of derivatives ∂vn/∂x cannot stay bounded above on the segment [0, 1/H] × {0}; that is, there exists a sequence (an) in [0, 1/H] such that ∂v (5) lim n (a , 0) = +∞. ∂x n

If we set xn = n + an − 1/(2H), (5) becomes

∂un 1 lim , 0 = +∞. ∂x 2H

Since ∂un/∂y(1/(2H), 0) = 0 by (2), the limit normal to the sequence of graphs + over (1/(2H), 0) is (−1, 0, 0). Therefore C (0) is a line of divergence for (un). By passing to a subsequence we can assume that (xn) is increasing, and clearly lim xn = +∞. This proves Lemma 11.

Conclusion of proof. Let (xn) be as in Lemma 11. Recalling the limit (3) and the surrounding notation, deﬁne

:= − c maxI0 (umax umin).

The set {(x, y) : umax(x, y) ≥ umin(x, y) + 2c} has a connected component W contained in ޒ+×[−1/(2H), 1/(2H)]. This component is unbounded. Now deﬁne

p 2 2 Wn = W ∩ (x, y) ∈ | x ≤ xn + 1/(4H ) − y .

The boundary of Wn is the union of ∂W ∩ Wn and 0n, the latter being the part + contained in the semicircle C (xn):

1 Wn H

xn ω = ω − ω Set e umax umin . Then Z Z Z = = + 0 eω eω eω. ∂Wn ∂W∩Wn 0n

By Lemma 2 in [Collin and Krust 1991], the integral on ∂W ∩ Wn is negative; and it decreases as n increases, since (xn) is increasing. Moreover, Z Z − = ≤ 0 < eω eω 2`(0n), ∂W∩Wn 0n 376 LAURENT MAZET where `(0n) is the length of 0n. Thus `(0n) is uniformly bounded away from 0. + Because of Lemma 11 and since 0n ⊂ C (xn), there exists a sequence (αn) in [0, 1] such that lim αn = 1 and Z ≥ ωumin αn`(0n). 0n

Finally, for n ≥ n0 > 0, we have Z Z Z Z − ≤ − = − ω ω ωumax ωumin ∩ e ∩ e ∂W Wn0 ∂W Wn 0n 0n

≤ `(0n) − αn`(0n) ≤ (1 − αn)`(0n) −−−−→ 0. n→+∞ But as we have seen the leftmost expression is strictly positive. This contradiction proves Theorem 10.

4. The uniqueness of López’s solutions

Theorem 12. If f : ޒ → ޒ is a continuous function that satisﬁes a lower ρt -circle condition, there is a unique solution of (CMC) on = ޒ×(−ht , ht ) agreeing with ϕ f on the boundary.

Proof. Take umin, umax, c, Ix , and W as in the proof of Theorem 10, but with 1/(2H) replaced by ht in the deﬁnition of Ix , and W. The limit (3) holds.

Lemma 13. There exists x0 ∈ ޒ+ such that f is monotonic on [x0, +∞). Proof. Consider the set ᏿ of points where f satisfies an upper 1/(2H)-circle condition. By the remark on page 372, ᏿ is nonempty and unbounded, and at each point of ᏿ the (possibly infinite) derivative of f has a well defined sign. + 0 We claim that there exists x1 ∈ ޒ such that f (x) has constant sign for every x ∈ ᏿ ∩ [x1, +∞). If not, take an increasing sequence of points in ᏿ tending to +∞ and such that f 0 is alternately positive and negative at these points. Then Lemma 9 yields an increasing sequence (cn) in ᏿ tending to +∞ and such that 0 f (cn) = 0 for all n. Let u be a solution of (CMC) on agreeing with ϕ f on ≤ 0 = the boundary. By Proposition 8, maxIcn u f (cn). Since f (cn) 0, Lemma 4 ≥ − − − ≤ − implies that minIcn u f (cn) (ρt t). Hence maxIcn (umax umin) ρt t, in contradiction with (3) since lim cn = +∞. This proves the claim. We assume that f 0(x) > 0 for x large, the case f 0(x) < 0 being handled similarly. Now suppose the assertion of the lemma fails, so there is an increasing sequence (an) in [x1, +∞) such that lim an = +∞ and f (an) is a local maximum of f for each n. Since f satisfies a lower ρt -circle condition, f is differentiable at every an. Let 0(s) be the circle of center (an − 1/(2H), s) and radius 1/(2H). For s large, 0(s) lies above the graph of f ; when s decreases down to the first contact UNIQUENESS RESULTS FOR CONSTANT MEAN CURVATURE GRAPHS 377 of 0(s) with the graph, we get a point x where f satisfies an upper 1/(2H)-circle 0 condition. By assumption f (x) > 0, so x ∈ [an − 1/(2H), an]. Let bn ∈ [x, an] 0 be a point maximizing f in [x, an]. Since f (x) > 0, we have bn ∈ (x, an], hence 0 f (bn) = 0. Using horizontal cylinders with 0(s) as vertical section, we prove that ≤ + ≤ + maxIbn u f (x) 1/(2H) f (bn) 1/(2H), where u a solution of (CMC) on 0 = ≥ agreeing with ϕ f on the boundary. Also, since f (bn) 0, we have minIbn u − − − ≤ + − = +∞ f (bn) (ρt t). Thus maxIbn (umax umin) 1/(2H) (ρt t). But lim bn , so this last inequality contradicts (3), proving the lemma.

We now assume that f is increasing on [x0, +∞); the case of f decreasing is handled similarly. By Theorem 5 in [Mazet 2006a], we know that f (x + 4/H) − f (x) cannot stay bounded as x goes to +∞. We even know that (6) lim f (x + 4/H) − f (x) = +∞. x→+∞ This identity will play the same role as (4) in the proof of Theorem 10.

+ Asymptotic behavior of umin. For a ∈ ޒ, denote by C (a) the arc of circle

p 2 2 2 2 2 {x ≥ a} ∩ x − a − (1/4H ) − ht + y = (1/4H ) .

Its endpoints are (a, ±ht ) and it contains the point (a + K, 0), where r 1 1 K = − − h2. 2H 4H 2 t Next we claim that Lemma 11 holds verbatim in this setting; that is, there exists + an increasing, diverging real sequence (xn) such that C (0) is an arc of divergence of the sequence (un) of translates of umin by (−xn, y). To see this, let vn be the translate deﬁned on by vn(x, y) = umin(x + n, y); its restriction to the boundary = + [ +∞ is ϕ fn , with fn(x) f (x n). For n large enough, fn is increasing on 1/H, ); z ϕ f

ᏺt

x 378 LAURENT MAZET using Proposition 7, we get vn(0, 0) ≤ fn(0) + 1/(2H). Now apply Lemma 4 at 4/H to see that the graph of vn lies above a nodoid ᏺt with horizontal axis in the vertical plane x = 4/H + A (0 ≤ A ≤ ρt since f is increasing). Since ᏺt lies below the graph, we have vn(4/H + A, 0) ≥ fn(4/H) − (ρt − t) (see ﬁgure on preceding page). Now translate ᏺt by the horizontal vector ex = (1, 0, 0); since fn is increasing, the nodoid ᏺt +sex does not cross the boundary, and so stays below the graph. Setting s = ρt − A we then get 4 4 v + ρ , 0 ≥ f − (ρ − t), n H t n H t which leads to 4 4 1 v + ρ , 0 − v (0, 0) ≥ f − f (0) − − ρ + t. n H t n n H n 2H t

By (6) we have lim vn(4/H + ρt , 0) − vn(0, 0) = +∞. Hence the sequence of derivatives ∂vn/∂x cannot stay bounded above on [0, 4/H +ρt ]×{0}; that is, there exists a sequence (an) in [0, 4/H + ρt ] such that ∂v (7) lim n (a , 0) = +∞. ∂x n

If we set xn = n + an − K , (7) becomes ∂u lim n (K, 0) = +∞ ∂x

Since ∂un/∂y(K, 0) = 0 by (2), the limiting normal to the sequence of graphs over + (K, 0) is (−1, 0, 0). Therefore C (0) is a line of divergence for (un), and the claim is proved (see end of proof of Lemma 11). To conclude the proof of our theorem (when f is increasing beyond x0) we simply repeat the reasoning in the last portion of the proof of Theorem 10, with the only difference that Wn is deﬁned as

p 2 2 p 2 2 Wn = W ∩ (x, y) ∈ x ≤ xn + 1/(4H ) − y − 1/(4H ) − ht .

(To deal with the case where f is decreasing we replace umin by umax and replace C+(a) by C−(a), deﬁned by

− p 2 2 2 2 2 C (a) = {x ≤ a} ∩ x − a + (1/4H ) − ht + y = (1/4H ) .

References

[Collin 1990] P. Collin, “Deux exemples de graphes de courbure moyenne constante sur une bande de R2”, C. R. Acad. Sci. Paris Sér. I Math. 311:9 (1990), 539–542. MR 92i:58041 Zbl 0716.53016 [Collin and Krust 1991] P. Collin and R. Krust, “Le problème de Dirichlet pour l’équation des surfaces minimales sur des domaines non bornés”, Bull. Soc. Math. France 119:4 (1991), 443–462. MR 92m:53007 Zbl 0754.53013 UNIQUENESS RESULTS FOR CONSTANT MEAN CURVATURE GRAPHS 379

[Courant and Hilbert 1962] R. Courant and D. Hilbert, Methods of mathematical physics, II: Partial differential equations, Wiley, New York, 1962. Reprinted 1989. MR 90k:35001 Zbl 0099.29504 [Delaunay 1841] C. Delaunay, “Sur la surface de révolution dont la courbure moyenne est constante”, J. Math. Pure Appl. 6 (1841), 309–320. [Eells 1987] J. Eells, “The surfaces of Delaunay”, Mathematical Intelligencer 9:1 (1987), 53–57. MR 88h:53011 Zbl 0605.53002 [Finn 1965] R. Finn, “Remarks relevant to minimal surfaces, and to surfaces of prescribed mean curvature”, J. Analyse Math. 14 (1965), 139–160. MR 32 #6337 Zbl 0163.34604 [Gilbarg and Trudinger 1983] D. Gilbarg and N. S. Trudinger, Elliptic partial differential equations of second order, 2nd ed., Grundlehren der Math. Wissenschaften 224, 1983. Corrected reprint, 1998. MR 2001k:35004 Zbl 0562.35001 [Huang 1995] W. H. Huang, “Boundedness of surfaces of constant sign mean curvature and the Phragmén–Lindelöf problem”, pp. 116–123 in Advances in geometric analysis and continuum mechanics (Stanford, 1993), edited by P. Concus and K. Lancaster, Int. Press, Cambridge, MA, 1995. MR 96g:53012 Zbl 0861.53010 [Hwang 1988] J.-F. Hwang, “Comparison principles and Liouville theorems for prescribed mean curvature equations in unbounded domains”, Ann. Scuola Norm. Sup. Pisa Cl. Sci. (4) 15:3 (1988), 341–355. MR 90m:35019 Zbl 0705.49022 [Hwang 1995] J. F. Hwang, “On the uniqueness of capillary surfaces over an infinite strip”, Pacific J. Math. 169:1 (1995), 95–105. MR 96f:58037 Zbl 0834.35050 [Jenkins and Serrin 1966] H. Jenkins and J. Serrin, “Variational problems of minimal surface type, II: Boundary value problems for the minimal surface equation”, Arch. Rational Mech. Anal. 21 (1966), 321–342. MR 32 #8221 Zbl 0171.08301 [López 2001] R. López, “Constant mean curvature graphs on unbounded convex domains”, J. Dif- ferential Equations 171:1 (2001), 54–62. MR 2002c:53011 Zbl 0987.35059 [López 2002] R. López, “Constant mean curvature graphs in a strip of ޒ2”, Pacific J. Math. 206:2 (2002), 359–373. MR 2003j:53010 Zbl 01868190 [Mazet 2006a] L. Mazet, “A height estimate for constant mean curvature graphs and uniqueness results”, Manuscripta Math. 119:2 (2006), 255–268. MR 2215971 Zbl 1093.53014 [Mazet 2006b] L. Mazet, “Lignes de divergence pour les graphes à courbure moyenne constante”, Ann. Inst. H. Poincaré Anal. Non Linéaire (2006). [Miklyukov 1979] V. M. Miklyukov, “On a new approach to Bernstein’s theorem and related questions for equations of minimal surface type”, Mat. Sb. (N.S.) 108:2 (1979), 268–290. In Russian; translated in Math. USSR Sb. 36 (1980), 251-271. MR 80e:53005 [Nitsche 1965] J. C. C. Nitsche, “On new results in the theory of minimal surfaces”, Bull. Amer. Math. Soc. 71 (1965), 195–270. MR 30 #4200 Zbl 0135.21701 [Serrin 1969] J. Serrin, “On surfaces of constant mean curvature which span a given space curve.”, Math. Z. 112 (1969), 77–88. MR 40 #3447 Zbl 0182.24001 [Serrin 1970] J. Serrin, “The Dirichlet problem for surfaces of constant mean curvature”, Proc. London Math. Soc. (3) 21 (1970), 361–384. MR 43 #1093 Zbl 0199.16604 [Spruck 1972/73] J. Spruck, “Infinite boundary value problems for surfaces of constant mean curvature”, Arch. Rational Mech. Anal. 49 (1972/73), 1–31. MR 48 #12329 Zbl 0263.53008 [Tam 1987a] L.-F. Tam, “On the uniqueness of capillary surfaces without gravity over an infinite strip”, Indiana Univ. Math. J. 36:1 (1987), 79–89. MR 88f:53011 Zbl 0678.49037 380 LAURENT MAZET

[Tam 1987b] L.-F. Tam, “On the uniqueness of capillary surfaces”, pp. 99–108 in Variational methods for free surface interfaces (Menlo Park, CA, 1985), edited by P. Concus and R. Finn, Springer, New York, 1987. MR 88c:53016 Zbl 0711.49060 [Wang 1990] A. N. Wang, “Constant mean curvature surfaces on a strip”, Paciﬁc J. Math. 145:2 (1990), 395–396. MR 91i:53016 Zbl 0724.49026

Received September 22, 2005.

LAURENT MAZET UNIVERSITE´ PAUL SABATIER,MIG LABORATOIRE E´ MILE PICARD, UMR 5580 31062 TOULOUSECEDEX 9 FRANCE [email protected] http://www.lmpt.univ-tours.fr/~mazet/ PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

GROUPS THAT ACT PSEUDOFREELY ON S2 × S2

MICHAEL P. MCCOOEY

A pseudofree group action on a space X is one whose set of singular orbits forms a discrete subset of its orbit space. Equivalently — when G is finite and X is compact — the set of singular points in X is finite. In this paper, we classify all of the finite groups which admit pseudofree actions on S2 × S2. The groups are exactly those that admit orthogonal pseudofree actions on S2 × S2 ⊂ ޒ3 × ޒ3, and they are explicitly listed. This paper can be viewed as a companion to a preprint of Edmonds, which uniformly treats the case in which the second Betti number of a four- manifold M is at least three.

1. Introduction

The classification of free actions of groups on spheres and products of spheres is an important problem in topology. In even dimensions, however, the question of which groups admit free actions is much less interesting: It follows from the Lefschetz fixed point theorem that such a group must admit a faithful, and rather special, representation on homology. Thus, for example, ޚ2 is the only group which can act freely on S2n, or freely and orientably on S2n × S2n. The Lefschetz fixed point theorem puts the same sort of restriction on free actions of groups on closed, simply-connected four-manifolds: in the homologically trivial case, there are none. This is one motivation for the concept of pseudofree actions. An action of a finite group G on a space X is pseudofree if it is free on the complement of a discrete set of points. (The use of this term is not consistent across the literature. Some authors also require that each singular point be fixed by the entire group.) Such actions make the singular set of the group action as small as is compatible with the Lefschetz theorem. It is a theorem of Edmonds [1997a] 4 that if b2(M ) ≥ 3, the only groups which can act pseudofreely on M and trivially 2 2 on H∗(M) are cyclic. In this paper, we treat the case M = S × S — which is especially interesting in its own right — for all possible actions on homology.

MSC2000: primary 55M35, 57S25, 57S17; secondary 20J06, 55T10. Keywords: pseudofree, group, group action, four-manifold.

381 382 MICHAEL P. MCCOOEY

Let G be a finite group with an action on S2 × S2 which preserves orientation and is locally linear and pseudofree. Two questions arise naturally: Which groups G admit such actions? And given a group which acts pseudofreely, can we classify the actions? A partial answer to the latter question is known: if G is cyclic, the Wilczynski–Bauer´ invariants [Bauer and Wilczynski´ 1996; Wilczynski´ 1994], detect whether two given pseudofree actions (on any simply-connected four- manifold) are topologically conjugate. The work of Edmonds and Ewing [1992], as well as that of other authors, addresses the question of which combinations of these invariants are realizable, but there are number-theoretical difficulties involved in making the answer explicit. Evidence from the study of other simple 4-manifolds (see [Cappell and Shane- son 1979; Kulkarni 1982] for the case of S4 and [Wilczynski´ 1987; Hambleton and Lee 1988] for ރP2) hints at an answer to the first question: the groups which admit pseudofree actions are exactly those which can also act pseudofreely and orthogonally on S2 × S2 ⊂ ޒ6. Our main result is that this is indeed the case: we classify the groups acting orthogonally and pseudofreely, and prove that these are the only groups admitting even locally linear pseudofree actions. (More precisely, the groups and their corresponding representations on homology are exactly those which occur in the linear case.) The remaining problem, that of classifying the actions themselves, is left for future work. Here is a brief outline of the paper: The homology representation of a group acting on S2 × S2 gives rise to a short exact sequence. Certain group extension problems related to this sequence are solved, and then the geometry of linear actions is used to determine which of these extensions admit pseudofree orthogonal actions. Actions are constructed in the course of the proofs. The classification of groups acting linearly and pseudofreely is stated explicitly in Theorem 3.9, and is reformulated in terms of the homology representation in Corollary 3.10. In Section 4, the conditions of Corollary 3.10 are shown to be necessary in the locally linear case, as well, thereby proving the main theorem. The proof uses a series of lemmas which apply orbit-counting arguments to the singular set of a group action, and then some group-theoretical and cohomological calculations to rule out minimal potential pathologies.

2. Group theory

2 2 2 2 Suppose a group G acts on S ×S . The induced action of G on H2(S ×S ) defines a representation ϕ : G → GL(2, ޚ). Since ϕ must respect the intersection form, it 2 2 must leave the positive and negative definite subspaces of H2(S × S ) invariant. + + − − H2 is spanned (rationally) by x y, and H2 is spanned by x y, where x and y represent the standard generators. With respect to this basis, ϕ(g) must have the GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 383 ±1 0 form for any g ∈ G. It follows that with respect to the standard basis, 0 ±1 1 0 0 1 ϕ(g) ∈ ± , ± =∼ ޚ × ޚ . 0 1 1 0 2 2 Thus the representation on homology induces a short exact sequence ϕ 1 → K → G → Q → 1, where K acts trivially on homology and Q ⊂ ޚ2 × ޚ2. This sequence allows us to approach general G-actions by first considering homologically trivial actions and then dealing with associated extension problems. In this section, we address the group theory involved in the extension problems, and also some cohomology calculations which will be needed later on in the nonlinear case. The results, though technical, are for the most part routine. First recall some generalities about the classification of group extensions: If K is an abelian group, and 1 → K → G → Q → 1 is an extension, then G acts on K by conjugation. Since K acts trivially on itself, conjugation induces a well-defined action of Q on K . For a specific g ∈ G, the conjugation automorphism −1 h 7→ g hg will be denoted µg. The action of Q defines a homomorphism ψ : Q → Aut K and a Q-module structure on K . As is well known, the extensions 1 → K → G → Q → 1 which give rise to the action ψ of Q on K are classified by H 2(Q; K ). If K is nonabelian, the situation is a bit more delicate: an extension problem 1 → K → G → Q → 1 with nonabelian kernel is described by the “abstract kernel” (K, Q, ψ), where ψ : Q → Out K describes the “outer action” of Q on K . For general (K, Q, ψ), an extension might not exist: ψ determines an obstruction cocycle in H 3(Q; Z(K )). The obstruction measures, roughly, whether it is possible to simultaneously realize the (outer) actions of each element of Q by conjugations in G. If the obstruction vanishes, then the extensions realizing (K, Q, ψ) are in (noncanonical) one-to-one correspondence with H 2(Q; Z(K )). For detailed accounts of this theory, see [Brown 1982] or [Mac Lane 1975]. In this section, we will mainly be concerned with extensions with quotient ޚ2. Assume such an extension exists, and pick some q ∈ G \ K . Two pieces of data determine the structure of G:

(1) The particular value in K of the square of q. Denote it by gq2 .

(2) The automorphism µq of K . In the abstract (when a specific extension is not given in advance), this automorphism will be denoted γ . 2 The choices of gq2 and γ are not arbitrary: γ must always fix gq2 , and γ must be an inner automorphism of K . In specific instances, there may be other restrictions as well. 384 MICHAEL P. MCCOOEY

Notation. The dihedral group of order 2m will be denoted Dm. Similarly, the − binary dihedral group of order 4m, with presentation << k, q | km = q2, q 1kq = −1 ∗ k >> , will be denoted Dm. (This group is also commonly referred to as a dicyclic or generalized quaternion group.) ∼ Lemma 2.1. Let K = ޚn. Each extension 0 → K → G → ޚ2 → 1 is of one of the following forms:

(1) ޚn o ޚ2, where the semidirect product automorphism is given by a certain

tuple (δ2, 2, . . . , pi , . . . , pk ). Every split extension is of this type.

+ × (2) (ޚm− o ޚ2n2 1 ) ޚm+ , where the semidirect product automorphism is inversion. ∗ (3) D n −1 × ޚm+ . 2 2 m− If n is odd, then every extension is split. Notation is explained in the proof.

Proof. We use additive notation for the group operation in ޚn. n p1 n pk = ··· ∼ n p × · · · × n p Let n p1 pq be a prime factorization. Then ޚn = ޚ 1 ޚ k , p1 pq and γ restricts to an automorphism of each of the factors. If p is odd, Aut ޚpn p is cyclic, so each order two automorphism of ޚpn p sends a n ∼ n generator to its inverse. However, Aut ޚ2 2 =ޚ2×ޚ2n2−2 , so if n2 >2, ޚ2 2 has three automorphisms of order two: x 7→ −x, x 7→ (1+2n2−1)x, and x 7→ (−1+2n2−1)x. | = = ± : Thus γ ޚ2n2 is encoded by a pair (δ2, 2), where δ2 1 or 0, 2 1, and (δ2, 2) x 7→ ( + δ · 2n2−1)x. For p > 2, we simply have γ | : x 7→ · x. With 2 2 ޚpn p p = this notation, we write γ (δ2, 2, . . . , pi , . . . , pk ). Then ޚn can be viewed as a product ∼ n ޚn = ޚ2 2 × ޚm− × ޚm+ , where ޚm− is the (odd) −1-eigenspace for γ , and ޚm+ is the (odd) +1-eigenspace. n p Since γ ﬁxes gq2 , gq2 ≡ 0 (mod p ) for each odd p with p = −1, so gq2 ≡

0 (mod m−). On the other hand, any element of ޚm+ is a multiple of 2, so q may be normalized to be trivial mod m+, as well. We may therefore assume that 0 gq2 ∈ ޚ2n2 × 0 × 0, and we have a subextension 1 → ޚ2n2 → G → ޚ2 → 1. 2 These are classiﬁed by H (ޚ2; ޚ2n2 ), where (δ2, 2) deﬁnes the module structure 2 on ޚ2n2 . Brown [1982, IV.4.2] computes that if δ2 = 1, then H (ޚ2; ޚ2n2 ) = 0, so every extension is split. Thus if δ2 = 1, the main extension 1 → ޚn → G → ޚ2 → 1 is a semidirect product. Henceforth we assume δ2 = 0. We now have 2 ∼ 2 ∼ H (ޚ2; ޚ2n2 ) = ޚ2, so H (ޚ2; ޚn) = ޚ2, as well. The split extensions are again semidirect products. But for each γ , there is one nonsplit extension. Since ޚm+ is central in G, the sequence can be written 00 n 0 → (ޚ2 2 × ޚm− ) × ޚm+ → G × ޚm+ → ޚ2 → 1, 00 so we may pretend for the moment that ޚm+ = {0}, and focus attention on G . GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 385

00 The structure of G depends on the sign of 2. If 2 = 1, then ޚ2n2 is central, 00 ∼ 00 ∼ ∗ and G = ޚm− ޚ n2+1 . If 2 = −1, then G = D n −1 . By appending the ޚm+ o 2 2 2 m− factor, we recover G in each case. Next we consider the possibility that K is a dihedral group. Since extensions of dihedral groups have nonabelian kernel, the algebra is a little more technical than in the cyclic case. Given an automorphism γ of Dn whose square is inner, there exists gq2 ∈ Dn 2 such that γ = µ , and this choice of g 2 is unique up to a factor in Z(D ). gq2 q n Z(Dn) is trivial if n is odd, and has order 2 if n is even. Clearly, gq2 is fixed by γ 2. Working through the definitions in [Brown 1982] or [Mac Lane 1975] shows that the obstruction to realizing the abstract kernel (Dn, ޚ2, γ ) as an extension vanishes exactly if gq2 is fixed by γ . It is straightforward to verify that vanishing of the obstruction depends only on the outer automorphism class of γ . S3 if n = 2, Claim 1. Aut Dn ≈ ޚn o Aut ޚn if n > 2. n 2 −1 We use the presentation Dn ≈ << s, t | s = t = 1, tst = s >> . If n = 2, then explicitly checking that the 2-cycle (s, t) and the 3-cycle (s, t, st) respect the group operation shows that any permutation of {s, t, st} defines an automorphism of D2. If n > 2 then, since an automorphism must send s to another element of order n, and t to an element which does not commute with f (s), any automorphism is a b of the form fa,b, where fa,b(t) = s t, and fa,b(s) = s , for a ∈ ޚn and (b, n) = 1. By calculating fa,b ◦ fc,d = fbc+a,bd , we see that Aut Dn is a semidirect product, as claimed. 2 We first deal with the case n = 2. Since D2 is abelian, γ = 1.

Case 1: γ is nontrivial. For convenience, we choose generators a, b of D2, and assume that γ transposes them, leaving ab ﬁxed. Since gq2 must be ﬁxed by γ , 2 gq2 = ab or 1; replacing gq by aq if necessary, we may assume q = 1. Then − G = << a, b, q | a2 = b2 = (ab)2 = q2 = 1, qaq 1 = b>> , or more simply, setting 4 2 −1 s = aq, t = b, G = << s, t | s = t = 1, tst = s >> , with D2 included as the subgroup << s2, t >> . Case 2: γ is trivial. In this case, G is abelian. It is straightforward to check that G = ޚ4 × ޚ2 or G = ޚ2 × ޚ2 × ޚ2. Assume henceforth that n > 2.

Claim 2. Inn Dn ≈ 2ޚn o {±1} ⊂ ޚn o Aut ޚn.

It sufﬁces to check that µsa = f2a,1, and µt = f0,−1. 2 2 2 Now let γ = fa,b. Then γ = fab+a,b2 . If γ ∈ Inn Dn, then b = ±1. But if 2 b = ±1, then b +1 must be a multiple of 2 in ޚn, so ab +a ∈ 2ޚn for any a ∈ ޚn. 386 MICHAEL P. MCCOOEY

Thus γ 2 is inner if and only if b2 = ±1. Since γ 2 must be inner for an extension to exist, we assume henceforth that b2 = ±1. Claim 3. If n is odd, the obstruction always vanishes. If n is even and b2 ≡ 1 (mod n), the obstruction vanishes unless a and (b2 − 1)/n are both odd in ޚ. If n is even and b2 ≡ −1 (mod n), the obstruction vanishes unless a and (b2 +1)/n are both odd in ޚ. Suppose n is even. Then the equation 2b0 = b + 1 has two solutions mod n. Fix one. The other is b0 + n/2. 0 0 2 2 0 ab a(b +n/2) Case 1: b = 1. In this case, γ = µ(sab ), so gq2 = s or s . If the first is 0 0 fixed by γ , so is the second. So we ask: when is γ (sab ) = sab ? ab0 bab0 0 Since γ (s ) = s , gq2 is fixed by γ if and only if (b − 1)ab ≡ 0 (mod n). Notice that 2(b − 1)ab0 = a(b2 − 1) ≡ 0 (mod n). However, it is possible that (b − 1)ab0 ≡ n/2 (mod n). This occurs if 2(b − 1)ab0 = a(b2 − 1) is an odd multiple of n, which occurs only if a and (b2 − 1)/n are both odd. 0 0 2 2 0 ab a(b +n/2) Case 2: b = −1. Now γ = µ(sab t), and gq2 = s t or s t. γ fixes gq2 = 0 sab t if and only if ab0 ≡ bab0 +a (mod n), if and only if (b−1)b0a+a ≡ 0 (mod n). As above, everything works unless 2((b −1)b0a +a) = (b2 −1)a +2a = (b2 +1)a is an odd multiple of n. This occurs only if a and (b2 + 1)/n are both odd. If n is odd, then similar but easier calculations show that gq2 is always fixed by γ . This establishes the claim.

Deﬁnition. Let us say that an automorphism γ of Dn is admissible if an extension 1 → Dn → G → ޚ2 → 1 exists with µk = γ for some k ∈ G \ Dn. 2 We have just seen that for n > 2, γ = fa,b is admissible if and only if b ≡ ±1 (mod n) and the obstruction mentioned in claim 3 vanishes. When extensions exist, they are in correspondence with 2 0 if n is odd, H (ޚ2, Z(Dn)) ≈ ޚ2 if n > 2 is even.

If n is even, the extensions correspond to the two choices of gq2 , which differ by the nontrivial element in Z(Dn). Thus ∼ n 2 −1 2 −1 −1 G = << s, t, q | s = t = 1, tst = s , q = gq2 , q sq = γ (s), q tq = γ (t)>> ∼ 2 −1 = << Dn, q | q = gq2 , q gq = γ (g) for g ∈ Dn >> .

This presentation depends on a particular γ ∈ [γ ] ∈ Out Dn. For some purposes, we might wish to normalize gq2 . To this end, note that for g ∈ Dn,

(µ ◦ γ )2 = µ ◦ γ 2 = µ . g gγ (g) gγ (g)gq2 GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 387

Thus gq2 can be modiﬁed by left multiplication by any element of the form gγ (g). The sequence will split when one of these candidates for gq2 is the identity. To summarize:

Lemma 2.2. Let γ ∈ Aut Dn. If γ is admissible, there are extensions

1 → Dn → G → ޚ2 → 1 in which the conjugation action of some q ∈ G \ Dn is given by γ . If n = 2, the extensions are 1 → D2 → D4 → ޚ2 → 1, 2 with D2 mapping to << s , t >> ⊂ D4, and the two abelian extensions with G ≈ ޚ4 ×ޚ2 3 and G ≈ (ޚ2) . For n > 2, there is exactly one such extension if n is odd, and two if n is even, and every extension is of this form. The resulting groups G have presentations of the form

2 −1 << Dn, q | q = gq2 , q gq = γ (g) for g ∈ Dn >> . Let Tet, Oct, and Icos denote the three symmetry groups of the Platonic solids. Lemma 2.3. Let K be one of the groups Tet, Oct, or Icos. Then every extension

1 → K → G → ޚ2 → 0 is split.

Proof. Recall that Tet ≈ A4, Oct ≈ S4, and Icos ≈ A5. A group G is said to be complete if Z(G) is trivial and every automorphism of G is inner. For n 6= 2, 6, Sn is complete; see [Rotman 1995, Theorem 7.5]. Alternating groups are never complete: for n > 2, An ⊆ Sn, and conjugation by the transposition (12) is an automorphism of An which is not inner in An.

Claim. Aut A4 ≈ S4.

We know S4 ⊆ Aut A4, since any conjugation by an element of S4 leaves A4 3 2 invariant, and since Z(S4) is trivial. A4 has the presentation << x, y | x = y = (xy)3 = 1>> . Any automorphism must send x to an element of order 3, and y to an element of order 2. Since A4 contains 8 elements of order 3 and 3 elements of order 2, we see that |Aut A4| ≤ 3 · 8 = 24. But |S4| = 24.

Claim. Aut A5 ≈ S5. This follows by an argument similar to the preceding one: the presentation 2 5 3 << x, y | x = y =(xy) =1>> gives an upper bound |Aut A5|≤180. But Aut A5 ⊇ S5, a group of order 120. Thus Out Tet ≈ Out Icos ≈ ޚ2, with the outer automorphism realized by conjugation by a transposition in S4 or S5. Out Oct is trivial, since S4 is complete. 388 MICHAEL P. MCCOOEY

Now, each of Tet, Oct, and Icos has trivial center, so in each case,

2 H (ޚ2; Z(K )) = 0. Thus if an abstract kernel admits a realization, it will be unique. Extensions do ∼ ∼ exist in all cases: the two extensions of Tet have G = S4 and G = Tet × ޚ2; the ∼ ∼ extensions of Icos have G = S5 and G = Icos × ޚ2; and the unique extension of ∼ Oct has G = Oct × ޚ2. All are split extensions. The last classiﬁcation lemma is somewhat more technical than the others, but it turns out to be exactly what is necessary when ϕ(G) = ޚ2 × ޚ2. ∼ ϕ Lemma 2.4. Suppose << a, b>> = ޚ2 × ޚ2, and 1 → ޚn → G → << a, b>> → 1 is = −1 = −1 = exact. Let Ga ϕ ( << a>> ) and Gb ϕ ( <> ). Finally, let γ µqb , viewed | = as an automorphism of Ga. Using the notation of Lemma 2.1, we have γ ޚn → → → → (δ2, 2, . . . , pi , . . . , pk ). If Ga is abelian and 1 Ga G <> 1 does not split, then one of the following holds: ∼ ∗ ∼ −1 G = D n × ޚ with = − G = ޚ and q = q (1) 2 2 m− m+ , 2 1, a 2n, γ ( a) a .

∼ + × ∼ × = (2) G = ((ޚm− o ޚ2n2 1 ) ޚm+ ) o ޚ2, with Ga = ޚa ޚ2, n2 > 1, 2 1, and γ (qa) = ((n/2)k)qa. ∼ ∗ ∼ (3) G = (D n −1 × ޚm+ ) × ޚ2, with Ga = ޚn × ޚ2, 2 = −1, and γ (qa) = qa. 2 2 m−

∼ + × × ∼ × = = (4) G = (ޚm− o ޚ2n2 1 ) ޚm+ ޚ2, with Ga = ޚn ޚ2, 2 1, and γ (qa) qa.

In the latter two cases, qa and qb commute. Notation, including some important normalizations for qa and qb, is explained in the proof. Proof. We use additive notation for the group operation in K , but multiplicative notation in the (possibly nonabelian) group G. Let k generate K = ޚn. G is generated by k, together with qa and qb, where ϕ(qa) = a and ϕ(qb) = b. We assume qa and qb are normalized so that g 2 and g 2 lie in the Sylow 2-subgroup qa qb of K , and

(1) g 2 is either 0, or a generator of this subgroup. qa (2) If 2 = 1, then g 2 is either 0, or a generator. qb (3) If 2 = −1, then either g 2 = 0, or it has order two. qb With this in mind, the problem easily reduces to the case n = 2n2 , since the restriction of γ to the subgroup ޚm−m+ of ޚn is known. We assume henceforth that n is a power of 2. If g 2 = 0, the sequence splits, so we assume g 2 6= 0. Once Ga is known, G is qb qb determined by the automorphism γ of Ga. In the proof of Lemma 2.1, we observed | ∈ { }×{± } 7→ + · n2−1 that γ ޚn is described by a pair (δ2, 2) 0, 1 1 , where k (2 δ2 2 )k, and also that if δ2 = 1, then 1 → ޚn → Gb → ޚ2 → 1 splits. So we may assume GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 389

γ (k) = 2 · k and consider the various cases for γ (qa). Note that γ (qa) = cqa for ∈ 2 = + n2−1 | some c K , and that γ (qa ) 2c gq2 . Hence c is determined mod 2 by γ ޚn . ∼ a If Ga = ޚ2n, then the argument cited in Lemma 2.1 shows that 1 → Ga → ±1 G → <> → 1 splits if c 6= 0. So we assume γ (qa) = qa , depending on 2. If = − =∼ ∗ = =∼ × −1 2 = 2 1, then G D2n2 . If 2 1, then G ޚ2n ޚ2, and (qaqb ) 1, so 1 → Ga → G → <> → 1 splits. ∼ If Ga = ޚn × ޚ2, we have several cases: ∼ (1) If c = 0 and 2 = 1, then G = ޚ2n2+1 × ޚ2. G contains three elements of order 2, but all are contained in Ga, so the sequence 1 → Ga → G → <> → 1 does not split. (2) If c = 0 and = −1, then G =∼ G × ޚ =∼ D∗ × ޚ . 2 b 2 2n2−1 2 − n2 1 ∼ n ∼ (3) If c =2 k and 2 =1, then Gb =ޚ2 2 +1, and G = Gboޚ2 =ޚ2n2+1 oޚ2, with −1 n2−1 qa qbqa = ((1 + 2 )k)q. Although G contains involutions, the extension 1 → Ga → G → <> → 1 does not split if n2 > 1. n −1 2 (4) If c = 2 2 k and 2 = −1, then (qaqb) = 1, so the sequence splits.

In Section 4, we will require explicit descriptions of the restriction maps r ∗ : 2 2 H (D4; ޚ) → H (H; ޚ) as H ranges over the various subgroups of D4. Rather than interrupt the flow of that argument later, we discuss them here. The methods used to calculate these maps are described in [Pearson 1996]. Some of the specific maps are also described there, and most of the rest were worked in the course of a conversation with the author of that paper. ∼ 4 2 −1 Using the presentation D4 = << s, t | s = t = 1, tst = s >> , the subgroups (up to conjugacy) can be enumerated as follows, with subscripts denoting generators: Gs ≈ ޚ4, Gs2 ≈ ޚ2, Gt ≈ ޚ2, Gst ≈ ޚ2, Gs2,t ≈ ޚ2 × ޚ2, and Gs2,st ≈ ޚ2 × ޚ2. The integral cohomology of these groups is computed from (known) descriptions of the cohomology with ޚ2 coefficients using the Bockstein spectral sequence. 1 ∼ 0 2 For a general group K , H (K ; ޚ2)=hom(Kabޚ2), and generators of H (K ; ޚ2) are often products of 1-dimensional classes. When integral cohomology classes are lifts of powers of ޚ2-classes, we will name them accordingly, even when the integral classes themselves are indecomposable. 1 Generators of H (D4; ޚ2) ≈ hom(H1(D4), ޚ2) are given by e and f , where e(s) = 1, e(t) = 0, f (s) = 1, and f (t) = 1. This seemingly asymmetrical choice of generators yields the convenient relation e∪ f =0, while e2 and f 2 lift to generators 2 2 2 eb and fb of H (D4; ޚ) ≈ ޚ2 ×ޚ2.(H2(D4; ޚ2) also contains an indecomposable element w which does not lift.) Similarly, for each copy of ޚ2 × ޚ2 described by an ordered set of generators 2 2 2 << x, y>> , we have H (ޚ2 ×ޚ2; ޚ) ≈ ޚ2 ×ޚ2 = << ab, bb >> , where a(x) = 1, a(y) = 0, 390 MICHAEL P. MCCOOEY b(x) = 0, and b(y) = 1. A word of warning: we use the same notation for H 2- generators of different copies of ޚ2 ×ޚ2; meaning can be determined from context. 2 Finally, H (ޚ4; ޚ) ≈ ޚ4, with a generator denoted by c. Both c and its reduction mod 2 are indecomposable. ∗ 2 2 Lemma 2.5. The restriction maps r : H (D4; ޚ) → H (H; ޚ), as H ranges over subgroups of D4, are as follows:

Gs2,t Gs2,st Gs2 Gt Gst Gs

eb2 0 bb2 0 0 bb2 2c fb2 bb2 0 0 bb2 0 2c Proof. We use the commutative diagram

∗ 2 2 2 r 2 << eb, fb >> ≈ H (D4; ޚ) −−−→ H (H; ޚ)     y y ∗ 2 2 2 r 2 << e , f , w>> ≈ H (D4; ޚ2) −−−→ H (H; ޚ2), where the vertical maps are induced by the coefﬁcient reduction. The right-hand 2 ∼ map is an isomorphism for all the subgroups except for ޚ4, where H (ޚ4; ޚ2) = 2 2 2 H (ޚ4; ޚ) ⊗ ޚ2. The fact that e and f are squares of one-dimensional classes makes calculation easy for every column except Gs. In that case, since the gen- ∗ erator of H2(ޚ4; ޚ2) is indecomposable, the map r : H2(D4; ޚ2) → H2(ޚ4; ޚ2) is zero. It follows immediately that each of r ∗(eb2) and r ∗( fb2) is either 0 or 2c. Determining which actually occurs requires a closer look at the Bockstein spectral sequence. Details are in [Pearson 1996]; our calculation later on actually only requires that each be an even multiple of c.

3. The linear case

Let us say that the group of linear actions on S2 × S2 is W = {A ∈ SO(6) | A(S2 × S2) = S2 × S2}. What is the structure of W? The homology representation ϕ extends to all of W. We will apply it in the proof of the following: ∼ Lemma 3.1. W = (SO(3) × SO(3) × ޚ2) o ޚ2, where the semidirect product automorphism is a coordinate switch in the factors of SO(3) × SO(3). Proof. The main claim that needs to be established is the exactness of the sequence

i ϕ 1 → SO(3) × SO(3) → W → ޚ2 × ޚ2 → 1. Since a rotation in SO(3) is homotopic to the identity, any g ∈ SO(3)×SO(3) is in the kernel of ϕ. For the reverse inclusion, suppose ϕ(g) = e ∈ ޚ2 × ޚ2. Since g GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 391 acts trivially on homology, it has a fixed point (x, y) ∈ S2 × S2. A fairly standard 2 2 calculation shows that the only sections of T(x,y) S × S with sectional curvature K = 1 are those tangent to the two factors. Since g is an isometry, it preserves the 2 2 2 2 splitting of T(x,y) S × S , so it acts on T(x,y) S × S by a pair of rotations θ1 and θ2. Thus g agrees with an element of SO(3)×SO(3) on a 5-dimensional subspace of ޒ6. Since det(g) = 1, g ∈ SO(3) × SO(3). This finishes the proof of exactness. Now, the quotient ޚ2×ޚ2 is easily seen to be generated by α, the isometry which acts by the antipodal map in each factor, and σ, which switches coordinates. α is central in SO(6), so << α>> extends SO(3) × SO(3) with a direct product. However, σ is not central, so the product is only semidirect. For the remainder of this section, let G be a finite subgroup of W which acts linearly and pseudofreely and preserves orientation. It is well known (see [Wolf 1984; Kulkarni 1982], for example, or Lemma 4.4 below) that the finite groups which act pseudofreely on S2 are either cyclic, dihedral, or one of the three symmetry groups of the Platonic solids. These groups also act pseudofreely on S2 × S2 via the diagonal action. It will turn out that if the induced action on homology is trivial, this is essentially, but not exactly, the only possibility. If the homology action is nontrivial, things are more complicated, and we approach them via the ϕ short exact sequence 1 → K → G → Q → 1. 2 2 2 Example. Consider the action of ޚ5 on S ×S defined by γ ·(x, y) = (γ ·x, γ ·y), where γ acts on S2 by a rotation of 2π/5 around an axis. The action has four isolated fixed points. Notice also that this action resembles the diagonal action, in the sense that γ · (x, y) = (γ · x, ψ(γ ) · y), where ψ is the automorphism of ޚ5 sending γ 7→ γ 2. However, this action is not equivalent to the diagonal action: A neighborhood of a singular point in the quotient by the diagonal action is a cone on the lens space L(5, 1), while the corresponding neighborhood for this action is a cone on L(5, 2). Definition. Suppose G acts on a space X. An action θ of G on X × X will be called semidiagonal if there is an automorphism ψ of G so that θ is equivalent to (g,(x, y)) 7→ (gx, ψ(g)(y)). Proposition 3.2. If G acts trivially on homology, then G is a polyhedral group, and its action is semidiagonal. Proof. Since G acts trivially on homology, we must have G ⊂ SO(3) × SO(3). Let (g, h) ∈ G. Since each of g and h preserves orientation on S2, each has a fixed point on S2. By the assumption of pseudofreeness, each fixed set must be 0-dimensional unless (g, h) = (e, e).

Claim. Let π1 and π2 denote the two obvious projections from SO(3) × SO(3) → SO(3). Then π1(G) ≈ π2(G). 392 MICHAEL P. MCCOOEY

Observe that for any g, there is a unique h so that (g, h) ∈ G. For if (g, h1) and 6= −1 −1 = −1 ∈ (g, h2) are both in G and h1 h2, then (g , h1 )(g, h2) (1, h1 h2) G. But −1 6= 2 −1 2 × 2 ∼ h1 h2 1, so it has two ﬁxed points on S . But then Fix((1, h1 h2), S S ) = S2 × S0, contradicting pseudofreeness. Now we can deﬁne ψ : π1(G) → π2(G) by

g 7→ the unique h such that (g, h) ∈ G.

ψ is clearly a homomorphism, and by symmetry, an isomorphism. This proves the claim. To simplify notation, write G1 = π1(G) and G2 = π2(G). 2 G1 acts pseudofreely on S , so it must be a polyhedral group. Moreover, the projection G → G1 must be injective, since if (g, h) ∈ ker π1, then g = e, and h = ψ(g) = e, also. Similar considerations apply to G2. Now, any two isomorphic finite subgroups of SO(3) are conjugate [Wolf 1984, Theorem 2.6.5]. That is, there is a change of coordinates with respect to which we actually have G1 = G2. Without loss of generality, assume we have applied it. (In doing so, we have fixed, once and for all, a particular identification between the two factors of S2 × S2. For later reference, this also defines a particular choice of coordinate switch σ : (x, y) 7→ (y, x). Abstractly, σ is only well-defined mod {e} × SO(3).) Then for any g ∈ G, we have g · (x, y) = (gx, ψ(g)y).

Here it is worthwhile to note a consequence of the proof of Lemma 2.3: If the automorphism ψ of G above is inner, then after applying the coordinate change, we may treat ψ as identity. Since Out Oct is trivial, every Oct action is diagonal. And since Out Tet ≈ Out Icos ≈ ޚ2, each of Tet and Icos admits at most one nondiagonal action. In fact, by embedding Tet in Oct, we see that the outer automorphism of Tet is realized by an SO(3) conjugation, so every Tet action is diagonal. However, if there were an SO(3) conjugation which realized the outer automorphism of Icos, then S5 would be a subgroup of SO(3). It isn’t, so there are two linear, pseudofree actions of Icos which are not linearly equivalent. If the action of G on homology is nontrivial, what can we say? If G ⊂ O(3) × −1 0 O(3), for example, the only possible nontrivial action is via 0 −1 . Thus we have a short exact sequence

1 → K → G → ޚ2 → 0.

Since we know the possible groups K in the above sequence, we should expect the possible groups G simply to be extensions of polyhedral groups by ޚ2. But it turns out that many of the extended actions can not be pseudofree and must be ruled out. The following statement describes the possible groups G. Explicit actions are constructed in its proof. GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 393

≈ −1 0 Proposition 3.3. If the homology action of G is given by ϕ(G) 0 −1 , then G ≈ ޚn × ޚ2 or G ≈ ޚ2n. Pseudofree actions exist in these cases. Proof. We begin with some observations about the geometry of W. Suppose g and s are nontrivial elements of SO(3). Each has a well defined axis l of rotation — we denote them respectively by Xg and Xs. Observe that µg(s) = s for some l if and only if g leaves Xs invariant. If g fixes Xs, then Xg = Xs, and µg(s) = s. If g inverts Xs, then Xs ⊥ Xg, and g is an order 2 rotation. In this case, −1 µg(s) = s . ∈ = −1 0 = Lemma 3.1 tells us that any q W with ϕ(q) 0 −1 has the form q (α, α)(r1, r2), where (r1, r2) ∈ SO(3) × SO(3). Also, an element S ∈ W acting homologically trivially and pseudofreely has the form (s1, s2), where neither coor- 2 dinate is 1. It follows from the previous observations that if q = 1 and µq (s) 6= s for some s, then r1 and r2 are both nontrivial order 2 rotations, and q fixes a torus. = −1 0 2 = So if G acts pseudofreely, any q with ϕ(q) 0 −1 and q 1 must be central in G. Similar considerations show that a q of order greater than 2 leaves invariant a unique axis in each factor, so at most one cyclic subgroup of K can be normalized by q, and in fact, the generator of such a group commutes with q. Thus for each q ∈ G \ K , one of the following holds:

(1) q has order 2, and µq is trivial. (2) q has order greater than 2, and q normalizes (at most) a single maximal cyclic subgroup of K , which it centralizes.

Now consider the possibilities for K : ∼ ∼ Case 1: K = ޚn. The only groups G satisfying conditions (1) and (2) are G = ∼ ޚn × ޚ2 and G = ޚ2n. If n is odd, these two groups are isomorphic, and the extension is realized by choosing q = (α, α) ◦ (1, r), where r has order 2 and shares an axis with the generator of K . If n is even, this construction realizes the case ޚn ×ޚ2. ޚ2n is realized if both r1 and r2 are order 2n rotations. Note that if n n is odd, the latter construction still yields a ޚ2n action, but if n is odd, then q fixes a torus. ∼ n 2 −1 Case 2: K = Dn = << s, t | s = t = 1, tst = s >> . If some q exists satisfying condition (2), then tq has order 2 but fails to satisfy (1). If no q satisfies (2), then n = 2, and either q, sq, tq, or stq has both r1 and r2 nontrivial, and thus fixes a torus. ∼ Case 3: K = Tet, Oct, or Icos. By Lemma 2.3, the sequence 1 → K → G → ޚ2 → 1 splits, so we may pick q of order 2. By condition (1), µq is trivial. Now let g ∈ K be an involution. Then gq also has order 2, but µgq is nontrivial, contradicting condition (1). 394 MICHAEL P. MCCOOEY

≈ 0 1 Next we consider the case in which ϕ(G) 1 0 . For actions which need not be locally linear, it follows from a theorem of Bredon that: → → → 0 1 → Proposition 3.4. (1) If the sequence 1 K G 1 0 1 splits, then G can not act pseudofreely. 2 2 ∼ (2) Suppose G acts on S × S with ϕ(G) = ޚ2. If the action of K = ker ϕ is pseudofree, and 1 → K → G → ޚ2 → 1 does not split, then the action of G is pseudofree. Proof. As a special case of Bredon’s theorem [1972, VII.7.5], an involution T on 2 2 ∗ 2 2 2 2 S × S with T 6= 1 on H (S × S ; ޚ2) has a ﬁxed point set F with H (F; ޚ2) = → → → 0 1 → ޚ2. If the sequence 1 K G 1 0 1 splits, then G contains such an involution. The second statement follows from the observation that for q ∈ G \ K , 2 Fix(q ) ⊆ Fix(q). ≈ 0 1 Corollary 3.5. If the homology action of G is given by ϕ(G) 1 0 , then K can’t be Tet, Oct, or Icos. Proof. This follows immediately from Lemma 2.3. ∼ ≈ 0 1 Proposition 3.6. If K = ޚn and ϕ(G) 1 0 , then either

∼ + × (1) G = (ޚm− o ޚ2n2 1 ) ޚm+ , or ∼ ∗ (2) G = D n −1 × ޚm+ , 2 2 m− with notation as in the proof of Lemma 2.1. Pseudofree linear actions exist in these cases.

Proof. We know from Proposition 3.4 that if 1 → K → G → ޚ2 → 1 splits, then G cannot act pseudofreely. It follows from Lemma 2.1 that G must be of type (1) or (2) (and that n is even). Recall from the beginning of Section 2 that the structure of G is determined by the data γ and gq2 . By Lemma 3.1, a hypothetical q ∈ G \ K has the form σr, where σ : (x, y) 7→ (y, x), and r = (r1, r2) ∈ SO(3)×SO(3). Our construction of a pseudofree G-action uses two ingredients: (1) A choice of a particular semidiagonal K -action, which makes it possible to realize γ as µq . 2 (2) An appropriate choice of r which ensures that q = gq2 . As in the proof of Proposition 3.3, conjugation by r must either ﬁx or invert the axis of rotation of K in each factor, so µσr = µr ◦ µσ is either (inversion) ◦ µσ , or simply µσ . For simplicity, we will assume that r1 and r2 share the axis of K , so µσr = µσ . The construction can also be carried out with minor modiﬁcations if ⊥ Xri X K . A semidiagonal embedding of K ⊂ SO(3) into SO(3) × SO(3) takes the form i k 7→ (k, ψ(k)) ∈ SO(3) × SO(3) for some ψ ∈ Aut K . GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 395

Observe that µσ (k, ψ(k)) = σ (k, ψ(k))σ = (ψ(k), k), and also that i(γ (k)) = 2 (γ (k), ψ(γ (k))). Since γ = 1, choosing ψ = γ allows us to realize γ as µq :

µq (i(k)) = µσ (k, ψ(k)) = (γ (k), k) = (γ (k), ψ(γ (k))) = i(γ (k)).

2 Next observe that q = σrσr = (r2r1, r1r2) = (r1r2, r1r2). Thus if we choose 2 r1 and r2 so that r1r2 = gq2 , then q = (gq2 , gq2 ) = (gq2 , ψ(gq2 )) = i(gq2 ) (recall that ψ = γ fixes gq2 ). It follows from the second part of Proposition 3.4 that the extended action is still pseudofree. ∼ ∼ 0 1 Proposition 3.7. If K = Dn and ϕ(G) = 1 0 , then G acts pseudofreely and linearly if and only if the sequence 1 → K → G → ޚ2 → 1 does not split. Proof. We know it is necessary that the sequence not split. To prove the converse, we assume it does not split and we construct a pseudofree G-action. Let the data γ and gq2 be given. As in the previous proof, q = σr, and we will choose r = (r1, r2) and ψ ∈ Aut Dn appropriately. In this case, choose any r1 and r2 so that r2r1 = ∈ ⊂ gq2 Dn SO(3), and each µri leaves Dn invariant. (For example, simply take = = = ◦ −1 ∈ × r1 1, r2 gq2 .) Let ψ µr2 γ Aut Dn, and let i embed Dn in SO(3) SO(3) i via g 7→ (g, ψ(g)). We show that the subgroup of W generated by i(Dn) and q is isomorphic to G, and then appeal to Proposition 3.4 to see that the resulting action of G is pseudofree. 2 For the first claim, it suffices to verify that q = i(gq2 ) and that µq ◦ i = 2 i ◦ γ . We have q = (r2r1, r1r2). On the other hand, i(gq2 ) = (r2r1, ψ(r2r1)) = −1 −1 = (r2r1, r2 (γ (r2r1))r2) (r2r1, r1r2), because γ fixes gq2 . Note that as a consequence, µ = µ = γ 2. r2r1 gq2 ∈ = = −1 = For g Dn, i(γ (g)) (γ (g), ψ(γ (g))) (γ (g), r2 gr2). But µq (i(g)) = = = −1 −1 −1 −1 = µq (g, ψ(g)) µr (µσ (g, ψ(g))) µr (ψ(g), g) (r1 r2 γ (g)r2r1, r2 gr2) 2 −1 −1 = −1 (γ (γ (g)), r2 gr2) (γ (g), r2 gr2).

Proposition 3.8. Suppose ϕ(G) = ޚ2 × ޚ2. Then ∼ ∗ G = D n × ޚ or (1) 2 2 m− m+ ,

∼ + × (2) G = ((ޚm− o ޚ2n2 1 ) ޚm+ ) o ޚ2, with n2 > 1, with notation as in Lemma 2.1. Pseudofree actions exist in these cases. = −1 0 = 0 1 ∈ 2 × 2 Proof. Let a 0 −1 and b 1 0 Aut H2(S S ), and suppose an extension ϕ 1 → H → G → << a>> × <> → 1

−1 −1 exists, with G acting pseudofreely. Let Ga = ϕ ( << a>> ), and Gb = ϕ ( <> ). 396 MICHAEL P. MCCOOEY

By Propositions 3.3 and 3.4, we know that K must be cyclic of even order, with ∼ Ga = ޚ2n or ޚn × ޚ2, and that the sequence 1 → Ga → G → <> → 1 must not split. Lemma 2.4 describes those groups G which satisfy these criteria. ∼ 2 If Ga = ޚn ×ޚ2, the geometry of W puts one more restriction on G. Let qa = 1, 2 2 where ϕ(qa) = a. Then qa = (α, α)(r1, r2), where (r1) = (r2) = 1. If both r1 and r2 are nontrivial, then qa ﬁxes a torus. If both are trivial, then the product of qa with the unique element of order two in K ﬁxes a torus. So exactly one of them is nontrivial. Suppose it is r1. The element qb has the form σ (s1, s2), so −1 = −1 qb qaqb (α, α)(1, s2 r1s2). Thus qa and qb can not commute in the linear case. Lemma 2.4 then shows that G necessarily has one of the forms given in the statement of the theorem. We must now construct pseudofree actions of these groups. For simplicity, we assume all rotations used in the constructions have the same axes. ∼ ∗ ∗ Case 1: G = D n × ޚm+ . We start with an action of Gb = D n −1 × ޚm+ as 2 2 m− 2 2 m− provided by Proposition 3.6. Extend ψ (the semidiagonalizing automorphism of 2 K ) to an automorphism of ޚ2n. Let qa = (α, α)(r, ψ(r)), where r is a generator ∼ of the Sylow 2-subgroup of K . Then << K, qa >> = ޚ2n acts pseudofreely. We need −1 −1 only verify that qb qaqb = qa to conclude that the subgroup of W generated by Gb and qa is isomorphic to G: −1 −1 = (s1 , s2 )σ (α, α)(r, ψ(r))σ (s1, s2) (α, α)(ψ(r), r) −1 −1 −1 = (α, α)(r , ψ(r) ) = qa .

∼ + × Case 2: G = ((ޚm− oޚ2n2 1 ) ޚm+ )oޚ2, with n2 > 1. Again, start with an action = + = of Gb (ޚm− oޚ2n2 1 )oޚm+ as in Proposition 3.6. Let qa (α, α)(r, 1), where r 2 ∼ is an order two rotation. Then qa = 1, so Ga = << K, qa >> = ޚn o ޚ2. And we have 2 qb = σ (s1, s2), chosen so that (qb) = k generates the 2-subgroup of ޚn. To show ∼ that << Gb, qa >> = G, we observe that

−1 n2−1 qa qbqa = (α, α)(r, 1)σ (s1, s2)(α, α)(r, 1) = 2 kqa, as required. Pseudofreeness of the extended actions follows from Proposition 3.4, as usual. Gathering together the results of this section, we have: Theorem 3.9. The following are all of the groups G which act linearly and pseudofreely on S2 × S2:

(1) Tet, Oct, Icos, ޚn, and Dn, acting homologically trivially. × ≈ ≈ −1 0 (2) ޚn ޚ2 and ޚ2n, where K ޚn, and ϕ(G) 0 −1 . (3) The groups GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 397

∼ + × (a) G = (ޚm− o ޚ2n2 1 ) ޚm+ , or ∼ ∗ (b) G = D n −1 × ޚm+ , 2 2 m− 2 −1 (c) << Dn, k | k = gk2 , kgk = γ (g) for g ∈ Dn >> , where 1 → Dn → G → ޚ2 → 1 does not split. ≈ 0 1 In this case, ϕ(G) 1 0 . (4) The groups ∼ ∗ G = D n × ޚ or (a) 2 2 m− m+ , ∼ + × (b) G = ((ޚm− o ޚ2n2 1 ) ޚm+ ) o ޚ2, with n2 > 1. In this case, ϕ(G) ≈ ޚ2 × ޚ2. This theorem can be restated in a form somewhat more amenable to generaliza- = −1 0 = 0 1 ∈ 2 × 2 tion to the nonlinear case: Let a 0 −1 and b 1 0 Aut H2(S S ), and suppose an extension ϕ 1 → K → G → Q → 1 exists, with Q ⊂ << a>> × <> . The sequence then determines the subgroups Ga = −1 −1 ϕ ( << a>> ), and Gb = ϕ ( <> ). Conversely, suppose a homology representation of a group G is described by a tuple (G, K, Ga, Gb) as above. 2 2 Corollary 3.10. (G, K, Ga, Gb) admits a linear, pseudofree action on S × S if and only if (1) K is polyhedral.

(2) If Ga 6= K , then Ga is abelian and K is cyclic.

(3) If Gb 6= K , then 1 → Ga → G → ޚ2 → 1 does not split. 2 (4) If ϕ(qa) = a, ϕ(qb) = b, and qa = 1, then qa and qb do not commute. Proof. We have already seen that condition 1 is necessary and sufficient in the homologically trivial case, and that condition 2 is necessary and sufficient in the ± 1 0 0 1 case. Condition 3 is necessary by Proposition 3.4 (Bredon’s theorem). On the other hand, this condition implies that 1 → H → Gb → ޚ2 → 1 does not split, and Propositions 3.6 and 3.7 show this to be sufficient in the 0 1 case. Finally, ∼ 1 0 Lemma 2.4 enumerates those groups G with ϕ(G) = ޚ2 ×ޚ2 which satisfy the first three conditions, and Proposition 3.8 shows that the last condition is necessary and sufficient to finish the classification.

4. The nonlinear case

In this section we prove that any group which acts pseudofreely and locally linearly also acts pseudofreely and linearly. (It should be pointed out that nonlinear actions definitely do exist: Edmonds and Ewing [1992] construct pseudofree actions of 398 MICHAEL P. MCCOOEY cyclic groups with fixed point data incompatible with linearity.) Our strategy follows the statement of Corollary 3.10: Bredon’s theorem shows that condition 3 is necessary. We show that conditions 1, 2, and 4 are necessary in the general case. For the remainder of the paper, we will make constant use of the Lefschetz fixed-point theorem and the Riemann–Hurwitz formula. We recall them here: Let g : X → X be a periodic, locally linear map on a compact manifold X. Then χ(X g) = λ(g), where λ(g), the Lefschetz number of g, is the alternating sum of g’s traces on homology. (The formula holds more generally, but this suffices for our purposes.) In particular, if g acts trivially on homology, χ(X g) = χ(X). In the context of homologically trivial pseudofree actions on S2 × S2, it follows that each element of G apart from the identity has exactly four fixed points. The Riemann–Hurwitz formula describes the orbit structure of a pseudofree action of a finite group G on a compact space X. If the action has singular orbits | | = | | = Gx1,..., Gxm; Gxi ni , and G N, then m X N χ(X) = Nχ(X/G) − N − . ni i=1 Again, the theorem generalizes — this time, to nonpseudofree actions. See [Kul- karni 1982], for example. In our case (X = S2 × S2; homologically trivial actions, for now), transfer considerations show that χ(X) = χ(X/G) = 4. Thus 4 N = . m 1 4 − P 1 − i=1 ni Definition. The Riemann–Hurwitz data for a pseudofree action is the m-tuple (n1,..., nm) described in the statement of the formula; each number ni is the size of the isotropy subgroup corresponding to one orbit. In the case of pseudofree actions of a group G on S2, it is easily seen that m ≤ 3, and the only possible Riemann–Hurwitz data are of the form (N, N), (2, 2, k), (2, 3, 3), (2, 3, 4), or (2, 3, 5). By local linearity, each isotropy group is cyclic, and then group theory calculations show that the only possible groups are cyclic, dihedral, tetrahedral, octahedral, or icosahedral (see [Kulkarni 1982] or Lemma 4.4 below). In contrast, in the case X = S2 × S2, an analogous calculation to the one in the S2 case only gives the bound m ≤ 7. There are some 20 or so infinite families of such m-tuples satisfying the Riemann–Hurwitz formula, plus a finite, but quite large, number of exceptional solutions. Also, the tuples only describe the sizes, and not the structures, of the isotropy groups. From this point of view, then, an argument directly analogous to the one in the S2 case would be intractable. However, the linear examples exhibit two more salient features: GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 399

(1) A priori, the isotropy groups might be any groups admitting free orthogonal actions on S3 (under the tangent space representation). In fact, they are all cyclic. (2) Each element of G has the four fixed points (x, y), (x, −y), (−x, y), and (−x, −y). In the cyclic, tetrahedral, and icosahedral cases, all four fixed points lie in different orbits, while in the dihedral and octahedral cases, the fixed-point sets of some elements meet two orbits in two points each. But all four fixed points of g ∈ G never lie in the same orbit. Thus in the list of isotropy groups corresponding to the Riemann–Hurwitz data, each isotropy group actually occurs two or four times. Because of this repetition, the data in this case correspond to the data for an action on S2, where things are simpler. Our strategy, then, is to show that in the general case, (1) The isotropy groups are still cyclic. (2) They still occur in pairs in the Riemann–Hurwitz data, and then appeal to the proof in the S2 case, to show: Theorem 4.1. A group acting pseudofreely, locally linearly, and homologically trivially on S2 × S2 is polyhedral. The proof will use a series of lemmas. We first describe the possible isotropy groups: Proposition 4.2. Suppose a finite group G acts pseudofreely, homologically trivially, and locally linearly on a closed, simply-connected four-manifold X with b2(X) ≥ 2. Then each isotropy group is cyclic. Proof. It is shown in [McCooey 2002] that, without the pseudofree assumption, such an isotropy group Gx0 must be abelian of rank 1 or 2. But a rank 2 group cannot act freely on the linking sphere to x0, and thus cannot act pseudofreely on X. Lemma 4.3. Let G and X be as in Proposition 4.2. For each singular point x, let o(X Gx ) be the number of G-orbits which meet X Gx . Then o(X Gx ) divides χ(X).

G Proof. Since Gx = << g>> for some g, |X x | = |Fix(g)| = λ(g) = χ(X). Observe that for any h ∈ G, if h−1gh(x) = x, then h(x) ∈ Fix(g). In other words, NG( << g>> ) acts on Fix(g). Two points of Fix(g) are in the same G-orbit if and only if they are in the same NG( << g>> )-orbit, for if gx = x, gy = y, and kx = y, −1 then k gk ∈ Gx = << g>> . But each NG( << g>> )-orbit of the action on Fix(g) has cardinality NG(g)/ << g>> . Since they all have the same size, the number of orbits must divide χ(X). 400 MICHAEL P. MCCOOEY

Now, corresponding to the set of singular orbits (and hence to the Riemann–

Hurwitz data (n1,..., nm)), there is a list orbit types Gx1 ,..., Gxn . We say the groups are repeated in pairs if each orbit type occurs in this list an even number of times. Lemma 4.4. Let G act homologically trivially and pseudofreely on S2 × S2. If the isotropy groups of the G-action are repeated in pairs, then G is polyhedral. Proof. Since the groups occur in pairs, m is even. Let m0 = m/2, and rearrange the Pm list so that G1 = Gm0+1, G2 = Gm0+2, and so forth. Since N(4− i=1(1−1/ni )) = 0 Pm 2 4, it follows that 2 = N(2 − i=1(1 − 1/ni )). As in the S case, the possible n1,..., nm0 are (N, N), (2, 2, k), (2, 3, 3), (2, 3, 4), and (2, 3, 5). In each case, these numbers represent the sizes of maximal cyclic subgroups of G, and each conjugacy class of maximal cyclic subgroups occurs in the list. Kulkarni [1982] describes the groups that can correspond to this data. Proofs are omitted for some of his assertions, and some details are different in our case and his, so we repeat the argument here. (N, N) clearly corresponds to a cyclic group of order N. For the remaining cases, we make the following observations: If a maximal cyclic subgroup << g>> of G is also normal, it has index ≤2 in G. For G/ << g>> operates freely on Fix(g), a set of four points. But since orbits come in pairs, Fix(g) must meet 2 or 4 G/ << g>> -orbits, so |G/ << g>> | = 2 or 1. By a similar argument, any maximal cyclic subgroup intersecting the center of G nontrivially has index ≤ 2. (2, 2, k) corresponds to a group of order 2k. It has a cyclic, index 2 subgroup << g>> of order k, which must be normal. Kulkarni assumes that his groups operate on a space with χ = 2, and uses this fact to prove that each h ∈ G \ << g>> must have order 2. In our case, we use the observation above. Now assume h ∈ G \ << g>> is ﬁxed. Since each (hga) has order 2, hgahga = 1, so hgah−1 = (ga)−1. It follows that G is dihedral. In the (2, 3, 3) case, |G| = 12. G is nonabelian. There are three nonabelian groups of order 12, and two of them contain elements of order 6. The third is Tet. In the (2, 3, 4) case, G has order 24 and trivial center. With this in mind, a look at a table of groups such as [Coxeter and Moser 1980, p. 137] easily shows that G = Oct. Finally, in the (2, 3, 5) case, |G| = 60. Sylow theory shows that G contains ﬁve copies of D2, intersecting trivially, 20 copies of ޚ3, and six copies of ޚ5. If any proper subgroup of G were normal, it would contain every conjugate of each element. Counting arguments show this to be impossible, so G is simple. Thus G = Icos. Lemma 4.5. Let G act pseudofreely, locally linearly, and homologically trivially on S2 × S2. Then the isotropy groups of the G-action are repeated in pairs. GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 401

Proof. Say G is nice if for each singular point x ∈ S2 × S2, (S2 × S2)Gx meets 2 or 4 G-orbits. If G is nice, then its isotropy groups are repeated in pairs. We assume inductively that every proper subgroup of G is nice, but that G is not. 2 2 Gx Then by Lemma 4.3, there is some x0 so that o((S × S ) 0 ) = 1; that is, G acts 2 2 G transitively on S × S ) x0 . = << >> = { } By Proposition 4.2, Gx0 is cyclic — say Gx0 g , and Fix(g) x0,..., x3 . Let h1(x0) = x1, h2(x0) = x2, and h3(x0) = x3. By minimality, G is generated by −1 g, h1, h2, and h3, and since hi ghi (x0) = x0, << g>> is normal in G. By minimality again, |g| = p for some prime p. Now, G/ << g>> acts freely on {x0,..., x3}, so | << >> | = × G/ g 4. Thus G is an extension of Gx0 by ޚ4 or ޚ2 ޚ2. The remainder of the proof is an analysis of these extensions. Most can be ruled out by elementary group theory considerations. The two more difﬁcult cases use arguments essentially due to Edmonds [1997b]. In the following cases, consideration of the possible automorphism actions of H on << g>> shows that some element of G \ << g>> must be central, and then that g is contained in a cyclic subgroup of order 2p, contradicting minimality.

(1) ޚp o (ޚ2 × ޚ2), for p > 2.

(2) Any nonsplit extension of ޚ2 by ޚ2 × ޚ2.

(3) ޚp o ޚ4, for p ≡ 3 (mod 4).

The case G = ޚ2 o ޚ4 must actually be abelian, so G is a direct product. ޚ2 × ޚ4 contains two cyclic subgroups of order 4. They intersect nontrivially, and therefore have the same fixed set. It follows that G must act semifreely, that is, each singular point is fixed by the entire group. Two cases remain: G = ޚ2 ×ޚ2 ×ޚ2, and G = ޚp oޚ4, where p ≡ 1 (mod 4). Suppose G = ޚ2 × ޚ2 × ޚ2 admits a nonnice action. G has seven cyclic sub- 3 groups. Since ޚ2 × ޚ2 does not act freely on S , their fixed-point sets are disjoint, and each has ޚ2 stabilizer. Since G is abelian, it acts on each fixed-point set, so each constitutes an orbit. The action has Riemann–Hurwitz data (2, 2, 2, 2, 2, 2, 2). Let X be S2 × S2 minus a small invariant neighborhood of the singular set, and let Y = X/G. Y is a compact 4-manifold with seven ޒP3 boundary components P1,..., P7. The cohomology long exact sequence for the pair (Y, ∂Y ) (with ޚ2- coefficients) shows that im(i ∗ : H 3(Y ) → H 3(∂Y )) has rank 6. The covering X → Y is classified by a map ϕ : Y → BG. This induces a map (ϕ ◦i)∗ : H 3(G) → 3S 7 3 H j Pj = (ޚ2) , which factors through H (Y ). Since it factors, the rank of im(ϕ ◦ i)∗ is at most 6. On the other hand, each π1(Pj ) maps to a different subgroup of π1(Y ) ≈ ޚ2 × ޚ2 × ޚ2 under the natural inclusion, and hence each Pj corresponds to a different ∼ nontrivial element of H1(Y ). Since H1(Y ) = hom(H1(Y ), ޚ2), each nontrivial 1 1 element of H (Y, ޚ2) restricts nontrivially to H (Pj ) for some j. By the Kunneth 402 MICHAEL P. MCCOOEY theorem and the cohomology structure of H ∗(ޒP3), each of these has a nonzero ∗ cube which maps to the top class of Pj . Thus rk(im(ϕ ◦ i) ) = 7, a contradiction. ∼ A somewhat similar argument covers the remaining case. Let G = ޚp oޚ4, with p ≡ 1 (mod 4). The semidirect product automorphism must have order 4; other- p 4 −1 wise G contains a cyclic subgroup ޚ2p. Thus G ≈ << g, h | g = h = 1, h gh = ga >> , where 4a = p + 1. G has p different subgroups of order 4, all of which are conjugate, so if an action exists, it has Riemann–Hurwitz data (4, 4, 4, 4, p). Define X and Y as before. Then Y has boundary consisting of five lens spaces : → L4, L4, L4, L4, and L p, with associated inclusions i1,..., i5 Ln j , Y . Once again, the covering X → Y is classified by a map ϕ : Y → BG, with induced maps ◦ : → ϕ i j Ln j BG. However, the cohomology calculation is just a bit subtler this time. For any coefficient module M, the transfer map gives an isomorphism

∗ ∗ ޚ4 H (G; M) → H (ޚp; M) . ∗ 1 With ޚp coefﬁcients, the ring H (ޚp) is generated by elements s ∈ H (ޚp) and 2 ∗ t ∈ H (ޚp), where t is the image of s under the Bockstein map. H (ޚp) therefore inherits a G-module structure from the action of G on s given by h · s = as. Thus 3 h·t = at, and h·st = (2a)st, so the action of h on H (ޚp) is given by multiplication 3 by −1. This has the unfortunate consequence that H (G; ޚp) = 0. To compensate ∗ for the G-action on H (ޚp), we replace ޚp with a twisted coefﬁcient module ޚfp, ∗ ∼ ∗ ∼ where h acts by −1. Note that H (ޚp; ޚfp) = H (ޚp; ޚp) = ޚp as ޚp-modules, since the restriction of the G-action to its subgroup ޚp is trivial. With this twisting, we observe: ∗ 3 3 ∼ (1) Restriction gives an isomorphism (ϕ ◦i5) : H (G; ޚfp) → H (ޚp; ޚfp) = ޚp. ∗ 3 3 (2) For j = 1,..., 4, the maps (ϕ ◦ i j ) : H (G; ޚfp) → H (ޚ4; ޚfp) = 0 are trivial. ∗ 3 4 Now, since the coboundary map δ : H (∂Y ; ޚfp) → H (Y, ∂Y ; ޚfp) is Poincare´ dual to the augmentation H0(∂Y ; ޚfp) → H0(Y ; ޚfp), we see that ∗ 3 3 im i : H (Y ; ޚfp) → H (∂Y ; ޚfp) 3 P consists of all (u1,..., u5) in H (∂Y ; ޚfp) such that u j = 0. In particular, if u1 + u2 + u3 + u4 = 0, then u5 = 0, as well. But by the observations above, 3 3 there are elements u ∈ H (Y, ޚfp) which restrict trivially to each H (L4; ޚfp), but 3 nontrivially to H (L p, ޚfp). This rules out the G-actions in question. Theorem 4.1 follows from the lemmas. Thus condition 1 in Corollary 3.10 is necessary in the general case. In other words, any group acting pseudofreely and homologically trivially on S2 × S2 also admits a linear, homologically trivial, pseudofree action. We now proceed to prove the necessity of condition 2. GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 403

2 × 2 = −1 0 Theorem 4.6. Let G act pseudofreely on S S , and suppose ϕ(G) 0 −1 . Then G is abelian, and ker ϕ is cyclic. Proof. Let K ⊂ G act homologically trivially, so that → → →ϕ −1 0 ≈ → 1 K G 0 −1 ޚ2 1 is exact. Claim. G/K acts freely on S2 × S2/K.

Let u ∈ G \ K , so that ϕ(u) generates ޚ2. If, for some x, u(x) lies in the same orbit as x, then for some k ∈ K , ku(x) = x. But ku has Lefschetz number 0, so if it has a fixed point, its fixed point set must be 2-dimensional. For the same reason, G/K acts freely on the set of singular points in S2 ×S2/K , and therefore it identifies the paired orbits of Lemma 4.5. Choose a small G- invariant open neighborhood N ⊂ S2×S2 of the singular set of the K -action, and let X = S2 × S2 \ N. Let Y = X/G. It follows from the claim that Y is a manifold with boundary consisting of two or three lens spaces Lni . (These ni ’s are exactly those which appear in pairs in the Riemann- Hurwitz data.) Since X is a cover of Y , we p,q ∼ p ; q ⇒ p+q can use the Cartan–Leray spectral sequence (E2 = H (G H (X)) H (Y )) to compute H 2(Y ; ޚ). Note that (1) Since G is finite, H 1(G; M) = 0 for any free ޚ-module M. (2) In general, H 0(G; M) =∼ M G, the submodule of M fixed by G. In our case, −1 0 2 0 ; 2 = G acts by 0 −1 on H (X), so H (G H (X)) 0. 2,0 ∼ 2 0 (3) No nonzero differentials enter or leave Ek = H (G; H (X)). 2 ∼ 2 Thus H (Y ) = H (G). Similar arguments show that, for each component Lni 2 ∼ 2 2 → 2 of ∂Y , H (Lni ) = H (ޚni ), and that the restriction H (Y ) H (Lni ) is given by the corresponding map on subgroups. By Poincare´ duality in Y, we have an exact sequence 2 2 ∼ H (Y ) → H (∂Y ) = H1(∂Y ) → H1(Y ). This becomes

r G i G 2 →⊕ M 2 ∼ M →⊕ H (G) H (ޚni ) = H1(ޚni ) H1(G). And since the restriction and inclusion maps factor through K , we also have a sequence r K i K 2 →⊕ M 2 ∼ M →⊕ H (K ) H (ޚni ) = H1(ޚni ) H1(K ). This sequence is not exact in general. However, from the previous sequence, it 2 G 2 K follows that H (∂Y )/im(r⊕ ) injects into H1(K ), and that H (∂Y )/im(r⊕ ) injects 404 MICHAEL P. MCCOOEY into a quotient of H1(K ). The remainder of the proof consists of cohomology calculations showing that this is impossible unless G is abelian and K is cyclic.

Case 1: K is cyclic. Since G maps onto ޚ2, [G, G] ⊆ K . Thus if G is nonabelian, the kernel of << k>> → H1(G) is nontrivial. In the exact sequence

r G i G 2 ⊕ 2 2 ∼ ⊕ H (G) → H (ޚn) ⊕ H (ޚn) = H1(ޚn) ⊕ H1(ޚn) → H1(G),

G 2 2 the image of r⊕ lies in the diagonal subgroup of H (ޚn) ⊕ H (ޚn). If G is nonabelian, the kernel of the inclusion map does not. Case 2: K = Tet, Oct, or Icos. To every polyhedral group K , there corresponds a binary polyhedral group Ke such that 1 → ޚ2 → Ke→ K → 1 is exact. The Lyndon– p,q ∼ p ; q ⇒ p+q Hochschild–Serre spectral sequence (E2 = H (K H (ޚ2)) H (Ke), in this case) relates the cohomologies of the groups in this sequence. It shows, in particular, that H 2(K ) injects into H 2(Ke). But since each Ke acts freely on S3, 2 2 Poincare´ duality shows that H (Ke) ≈ H1(Ke). Thus H (K ),→ H1(Ke). 2 By computing the sizes of H1(Ke) in each case, we ﬁnd that |H (Tet)| ≤ 3, | 2 | ≤ 2 = L 2 = × H (Oct) 2, and H (Icos) 0. On the other hand, H (ޚni ) n1 n2 × n3. Finally, H1(Tet) = ޚ3, H1(Oct) = ޚ2, and H1(Icos) = 0. In each case, 2 K H (∂Y )/im(r⊕ ) is too large to inject into a quotient of H1(K ).

Case 2: K = Dn. Recall that ( ޚ2 = << t >> if n is odd, H1(Dn) ≈ ޚ2 × ޚ2 = << s, t >> if n is even, and ( 2 ޚ2 if n is odd, H (Dn) ≈ ޚ2 × ޚ2 if n is even.

= = = L 2 = = Since n1 n2 2 and n3 n, we have H (ޚni ) 4n. Except in the cases n 2 2 K and n = 4, it is immediate that H (∂Y )/im(r⊕ ) cannot inject into any quotient of H1(Dn). We consider the remaining possibilities: Suppose n = 2. Using Lemma 2.2, we ﬁnd that the only nonabelian extension ∼ 4 2 −1 −1 1 → D2 → G → ޚ2 → 1 has G = D4 = << s, t | s = t = 1, tst = s >> , with 2 D2 = << s , t >> . Thus we have an exact sequence

D4 D4 2 r⊕ 2 2 2 ∼ i⊕ H (D4) → H (ޚ2)⊕ H (ޚ2)⊕ H (ޚ2) = H1(ޚ2)⊕ H1(ޚ2)⊕ H1(ޚ2) → H1(D4),

2 2 where the three ޚ2 subgroups are generated by s , t, and s t. Since the restrictions D4 D4 and inclusions factor through D2, Lemma 2.5 shows that |im(r⊕ )| = |ker i⊕ | = 2. This contradicts exactness. GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 405

D2 also has two abelian extensions: 1 → D2 → D2 × ޚ2 → ޚ2 → 1, and 1 → D2 → ޚ4 × ޚ2 → ޚ2 → 1. In each case, the restriction and inclusion maps can be explicitly calculated, and the sequence is seen not to be exact. D2 is the only candidate for ker ϕ which is abelian, but not cyclic. = = << >> ≈ = << >> ≈ Finally, suppose K D4. Then we have ޚn1 t ޚ2, ޚn2 st ޚ2, D = << >> ≈ 4 and ޚn3 s ޚ4. Consulting Lemma 2.5 again, we see that r⊕ has matrix 0 1 D4 0 1 1 1 0 relative to the bases given there; i⊕ has matrix , as is easily checked. 2 2 1 1 0 Now, if r G is not onto, then the element counts which applied for most n also D4 apply for D . But if r G is onto, then L H 2(ޚ )/im(r D4 ) should inject into 4 D4 ni ⊕ D4 D4 H1(D4), so we should have ker i⊕ ⊆ im(r⊕ ). However, the element (1, 1, 1) ∈ H1( << t >> ) ⊕ H1( << st >> ) ⊕ H1( << s>> ) is in the kernel, but not in the image. Condition 3 of Corollary 3.10 is necessary by Bredon’s theorem. A related result of Bredon helps us establish the necessity of condition 4, and thus complete the proof of the main theorem:

2 Proposition 4.7. Suppose G acts pseudofreely, ϕ(qa) = a, ϕ(qb) = b, and qa = 1. Then qa and qb do not commute.

Proof. Given the necessity of conditions 1, 2, and 3 of Corollary 3.10, it suffices ∼ to rule out the possibility that G = << qa, qb>> = ޚ2 × ޚ4k, where k ≥ 1. In the case of the linear models (Proposition 3.8), the argument divided into two parts: 2k If qa = (α, α), then qaqb fixes a torus, contradicting pseudofreeness. And if 2 2 qa = (α, ρ) or (ρ, α), then the fact that qb exchanges factors of S × S means that qa and qb cannot commute, so no action exists, even with a two-dimensional singular set. 2 Assume, then, that qa and qb commute. Now, qa must act freely, so X = S × 2 S / << qa >> is a manifold which inherits an action by << qb>> = ޚ4k. As motivation, we note that the linear models for qa are distinguished by the intersection forms 2 2 (with ޚ2 coefficients) of the quotient spaces: In X1 = S × S /(α, α), the diagonal 2 2 S maps to an embedded ޒP with self-intersection 1, so w2(X1) 6= 0. On the 2 2 other hand, generators of second homology for X2 = S × S /(α, ρ) are given by the images of S2 × ∗, (where ∗ is some point fixed by ρ), and (∗ ∪ −∗) × S2, (where ∗ is any point in the first factor). Each of these has trivial self-intersection, so w2(X2) = 0. To complete the proof, we will show (just as in the linear models) that if << qb>> acts on X, then w2(X) 6= 0. Bredon’s theorem then guarantees a two-dimensional singular set. 2 2 2 Let x and y denote the standard generators for H (S × S ; ޚ2), and consider the cohomology spectral sequence of the covering:

p,q = p ; q 2 × 2; ⇒ p+q ; E2 (X) H (ޚ2 H (S S ޚ2)) H (X ޚ2). 406 MICHAEL P. MCCOOEY

<< xy>> ޚ2 ޚ2 ޚ2 ޚ2 ޚ2 ޚ2 ··· 0 0 0 0 0 0

<< x, y>> ޚ2 ⊕ޚ2 ޚ2 ⊕ޚ2 ޚ2 ⊕ޚ2 ޚ2 ⊕ޚ2 ޚ2 ⊕ޚ2 ޚ2 ⊕ޚ2 ··· 0 0 0 0 0 0

<< 1>> ޚ2 ޚ2 ޚ2 ޚ2 ޚ2 ޚ2 ···

2 3 4 5 ޚ2 << s>> << s >> << s >> << s >> << s >> ···

Table 1. E2(X).

It follows from the multiplicative structure of the spectral sequence that the behavior of the entire E2 page is determined by d2(x) and d2(y). At least one must be nonzero, since the sequence converges to the cohomology of a four-manifold. And since << qb>> acts on the quotient, the differentials must respect the induced 2 2 2 3 action of qb on H (S × S ). Hence d2(x) = d2(y) = 1s , and ker d2 is generated by x + y. It is easy to check that E3 = · · · = E∞, so u = [x + y] is a permanent cocycle. This spectral sequence is identified by a homotopy equivalence with that of the 2 × 2 → × 2 × 2 → fibration S S Eޚ2 ޚ2 (S S ) Bޚ2 (see [Hu 1959, IX.15]), and 0,∗ under this identification, the cocycles which live to E∞ are those in the image of ∗ ∗ ∗ 2 2 2 p : H (X) → H (S × S ). Thus there is a class u ∈ H (X; ޚ2) which lifts to 2 2 2 x + y ∈ H (S × S ; ޚ2). We claim that u ∪ u 6= 0. To see this from the homological point of view, let 2 2 ∗ ∼ C∗(X) denote the singular chain complex of X. Then S ×S = eX, and C (eX; ޚ) = ∗ ∗ ∗ ∗ C (X; ޚ) ⊗ޚ ޚ[ޚ2]. The covering projection induces p : C (X) → C (eX) via c 7→c⊗(1+qa). Moreover, because C∗(X) is a free ޚ-module, there is a natural iso- ∗ ∗ ∗ ∗ morphism µ : C (X; ޚ)⊗ޚ2 → C (X; ޚ2), so that H∗(C (X)⊗ޚ2) = H (X; ޚ2). 2 2 2 Every class in H (S × S ; ޚ2) is integral. Thus we can choose a cochain ∗ ∗ 3 ˜ υ ∈ C (X; ޚ) so that δ(p (υ)) = δ(υ ⊗ (1 + qa)) = 0 in C (X; ޚ) and such 2 that υ ⊗1 is a cocycle representing u in H (X; ޚ2). (Note that υ itself need not be an integral cocycle. For related discussion, see [Acosta and Lawson 1997].) Then ∗ 2 ∗ 2 [p (υ) ⊗ 1] = x + y ∈ H (eX; ޚ2), and [p (υ)] ∈ H (eX; ޚ) represents an integral lift of x + y — say (2m + 1)xˆ + (2n + 1)yˆ. So [p∗(υ) ∪ p∗(υ)] = (8mn + 4m + 4n + 2)(xˆ ∪y ˆ) ≡ 2 mod 4. But p∗(υ)∪ p∗(υ) is also an equivariant cochain and hence must be of the form 4 γ ⊗ (1 + qa), where γ ∈ C (X; ޚ). So [γ ] ≡ 1 mod 2, and [γ ⊗ 1] represents u ∪ u. Thus u ∪ u 6= 0, so w2(X) 6= 0. A heuristic geometrical argument is more direct: the Poincare´ dual of u can be represented by a surface F, and Fe ⊂ S2 × S2 will represent an integral lift of GROUPS THAT ACT PSEUDOFREELY ON S2 × S2 407

PD(x + y), so Fe· Fe ≡ 2 (mod 4). The intersection points will be paired up by the << qa >> -action, so F · F ≡ 1 (mod 2), and u ∪ u 6= 0. Combining Corollary 3.10, Theorem 4.1, Theorem 4.6, Proposition 4.7, and Proposition 3.4, we have our main theorem: Theorem 4.8. Any ﬁnite group which admits a locally linear, orientation preserving, pseudofree action on S2 × S2 also admits a linear, orientation preserving, pseudofree action. In fact, the pairs (group, cohomology representation) are exactly those which occur in the linear case.

References

[Acosta and Lawson 1997] D. Acosta and T. Lawson, “Even non-spin manifolds, spinc structures, and duality”, Enseign. Math. (2) 43:1-2 (1997), 27–32. MR 98m:57033 Zbl 0889.57013 [Bauer and Wilczynski´ 1996] S. Bauer and D. M. Wilczynski,´ “On the topological classiﬁcation of pseudofree group actions on 4-manifolds, II”, K -Theory 10:5 (1996), 491–516. MR 97m:57017 Zbl 0864.57012 [Bredon 1972] G. E. Bredon, Introduction to compact transformation groups, Pure and Applied Mathematics 46, Academic Press, New York, 1972. MR 54 #1265 Zbl 0246.57017 [Brown 1982] K. S. Brown, Cohomology of groups, Graduate Texts in Mathematics 87, Springer, New York, 1982. MR 83k:20002 Zbl 0584.20036 [Cappell and Shaneson 1979] S. E. Cappell and J. L. Shaneson, “Pseudofree actions, I”, pp. 395– 447 in Algebraic topology (Århus, 1978), edited by J. L. Dupont and I. H. Madsen, Lecture Notes in Math. 763, Springer, Berlin, 1979. MR 81d:57034 Zbl 0416.57020 [Coxeter and Moser 1980] H. S. M. Coxeter and W. O. J. Moser, Generators and relations for discrete groups, 4th ed., Ergebnisse der Mathematik 14, Springer, Berlin, 1980. MR 81a:20001 Zbl 0422.20001 [Edmonds 1997a] A. L. Edmonds, “Homologically trivial group actions on 4-manifolds”, Preprint, 1997. math.GT/9809055 [Edmonds 1997b] A. L. Edmonds, “Pseudofree actions on spheres”, unpublished notes, 1997. [Edmonds and Ewing 1992] A. L. Edmonds and J. H. Ewing, “Realizing forms and ﬁxed point data in dimension four”, Amer. J. Math. 114:5 (1992), 1103–1126. MR 93i:57047 Zbl 0766.57020 [Hambleton and Lee 1988] I. Hambleton and R. Lee, “Finite group actions on P2(C)”, J. Algebra 116:1 (1988), 227–242. MR 89e:14044 Zbl 0659.20041 [Hu 1959] S.-t. Hu, Homotopy theory, Pure and Applied Mathematics 8, Academic Press, New York, 1959. MR 21 #5186 Zbl 0088.38803 [Kulkarni 1982] R. S. Kulkarni, “Pseudofree actions and Hurwitz’s 84(g−1) theorem”, Math. Ann. 261:2 (1982), 209–226. MR 84e:57034 Zbl 0479.57023 [Mac Lane 1975] S. Mac Lane, Homology, Grundlehren der Math. Wissenschaften 114, Springer, Berlin, 1975. MR 96d:18001 Zbl 0328.18009 [McCooey 2002] M. P. McCooey, “Symmetry groups of four-manifolds”, Topology 41:4 (2002), 835–851. MR 2003f:57070 Zbl 1022.57009 408 MICHAEL P. MCCOOEY

[Pearson 1996] K. Pearson, “Integral cohomology and detection of w-basic 2-groups”, Math. Comp. 65:213 (1996), 291–306. MR 96d:20056 Zbl 0848.20045 [Rotman 1995] J. J. Rotman, An introduction to the theory of groups, 4th ed., Graduate Texts in Mathematics 148, Springer, New York, 1995. MR 95m:20001 Zbl 0810.20001 [Wilczynski´ 1987] D. M. Wilczynski,´ “Group actions on the complex projective plane”, Trans. Amer. Math. Soc. 303:2 (1987), 707–731. MR 88j:57041 Zbl 0639.57022 [Wilczynski´ 1994] D. M. Wilczynski,´ “On the topological classiﬁcation of pseudofree group actions on 4-manifolds, I”, Math. Z. 217:3 (1994), 335–366. MR 96f:57014 Zbl 0813.57030 [Wolf 1984] J. A. Wolf, Spaces of constant curvature, 5th ed., Publish or Perish, Wilmington, DE, 1984. MR 88k:53002

Received November 1, 2004.

MICHAEL P. MCCOOEY DEPARTMENT OF MATHEMATICS FRANKLINAND MARSHALL COLLEGE LANCASTER, PA 17604-3003 UNITED STATES [email protected] http://edisk.fandm.edu/michael.mccooey/ PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

PROJECTABILITY AND UNIQUENESS OF F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES

FRANK MÜLLERAND SVEN WINKLMANN

We study immersed critical points X of an elliptic parametric functional = R ∧ Ᏺ(X) B F(Xu Xv) du dv that are spanned into a partially free boundary conﬁguration {0, ᏿} in ޒ3. We suppose that ᏿ is a cylindrical support surface and that 0 is a closed Jordan arc with a simple convex projection. Under geometrically reasonable assumptions on {0, ᏿}, F, and X we prove the projectability and uniqueness of stable immersions. This generalizes a result for minimal surfaces obtained by Hildebrandt and Sauvigny.

1. Introduction

It is well known that one cannot expect uniqueness for disc-type solutions of Plateau’s problem spanning an arbitrary closed Jordan curve 0 ⊂ ޒ3. However, 0 bounds exactly one minimal surface if it has a simple projection onto a planar convex curve; this is a celebrated theorem of Rado´ [1926], with a contribution by Kneser [1926]. Moreover, this surface must in fact be a graph. Sauvigny [1982] was able to generalize this result to surfaces with prescribed mean curvature under an additional stability assumption. More generally, Hildebrandt and Sauvigny studied the partially free boundary problem for minimal surfaces inside boundary conﬁgurations {0, ᏿}, consisting of a closed Jordan arc 0 with a simple convex projection and a cylindrical support surface ᏿. They proved various uniqueness results and the existence of graph representations; see [Hildebrandt and Sauvigny 1991; 1992; 1995]. Again, this result was extended in [Muller¨ 2005] to stable surfaces of prescribed mean curvature. Here we consider this partially free boundary problem for elliptic parametric functionals of the type Z Ᏺ(X) = F(Xu ∧ Xv) du dv, B

MSC2000: 53C42, 35J65, 49Q10. Keywords: F-minimal immersions, partially free boundaries, uniqueness, projectability, Wulff shape. Winklmann was ﬁnancially supported by the Deutsche Forschungsgemeinschaft.

409 410 FRANK MÜLLER AND SVENWINKLMANN whose integrand F : ޒ3 \{0} → ޒ+ represents a smooth elliptic Lagrangian satisfying the homogeneity relation

(1-1) F(tz) = t F(z) for all z ∈ ޒ3, t > 0.

Obviously, Ᏺ generalizes the classical area functional Z Ꮽ(X) = |Xu ∧ Xv| du dv B obtained in the case F(z) = |z|. Using sophisticated tools from the direct methods in the calculus of variations, Hildebrandt and von der Mosel [1999] studied Plateau’s problem for general elliptic parametric functionals of the form Z Ᏺ(X) = F(X, Xu ∧ Xv) du dv; B they also addressed the partially free boundary problem [2002]. For a detailed survey on the existence and regularity theory as well as further remarks on the literature, see [Hildebrandt and von der Mosel 2005]. Investigating the functional Ᏺ from a more geometric point of view, Winklmann [2003] and Clarenz and von der Mosel [2004] studied immersed critical points, the so-called F-stationary immersions, under Plateau-type boundary conditions. This leads to surfaces of vanishing or, more generally, prescribed anisotropic mean curvature, allowing extensions of Rado’s´ and Sauvigny’s projectability and uniqueness results. Here we obtain similar results for immersed surfaces with partially free boundaries. In particular, we extend the uniqueness result of [Hildebrandt and Sauvigny 1995] in an appropriate manner and prove graph representations for stable critical points, or F-stable immersions in short, in the cylindrical boundary configuration {0, ᏿}. (Concerning anisotropic capillary surfaces with free boundaries, see [Koiso and Palmer 2006].) Specifically, in Section 2 we formulate general assumptions and collect basic facts on F-stationary immersions with partially free boundaries; Lemma 2.5 might be of independent interest. In Section 3 we show that the free boundary remains inside 6 × ޒ (Lemma 3.1) and prove that the surface is transversal to the fixed boundary (Lemma 3.2). We also derive an equation for the surface normal at the free boundary (Lemma 3.3). In Section 4 we compute the second variation of Ᏺ (Theorem 4.1) and use the previous results to construct an admissible test function in the stability inequality (Lemma 4.2). In Section 5 we finally prove the projectability of F-stable immersions (Theorem 5.1). This leads to the desired F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES 411 uniqueness result (Theorem 5.2) via a comparison principle for mixed boundary value problems of minimal surface type.

2. Notation and preliminary results

A boundary configuration {0, ᏿} consists of a closed Jordan arc 0 ⊂ ޒ3 of class 3 3 3 C with endpoints P1, P2 and an embedded support manifold ᏿ ⊂ ޒ of class C such that ᏿ ∩ 0 = {P1, P2}. We also suppose that 0 meets ᏿ with a positive angle at these points. Definition 2.1. A boundary configuration {0, ᏿} is named projectable if 3 (a) ᏿ = 60 ×ޒ is a cylinder surface over the planar Jordan curve 60 ∈ C , which decomposes the x1, x2-plane E into two unbounded domains. (b) 0 is representable as a graph over E: 0 = {(x1, x2, γ (x1, x2)) : (x1, x2) ∈ 0}, where γ (x1, x2)∈C3(0) denotes the height function and 0 ∈C3 the projection of 0 onto E. 3 Let 60 ∈ C be parametrized by σ = σ (s), −∞ < s < +∞, with arc length 0 |σ (s)| ≡ 1. Suppose that Pk = (σ (sk), γ (σ (sk)), k = 1, 2, for −∞ < s1 < s2 < +∞. If we set 6 := {σ (s) : s1 < s < s2}, the closed Jordan curve 0 ∪6 bounds a simply connected domain G ⊂ E. Definition 2.2. A projectable boundary configuration {0, ᏿} is admissible if

(a) 0 is convex with respect to G and does not meet 60 perpendicularly, and

(b) for each s ∈ (−∞, s1) ∪ (s2, +∞), the normal line L(s) := {p ∈ E : hp − σ (s), σ 0(s)i = 0}

meets G ∪ 60 only at the point σ (s). We also introduce the tangent t(x) := (σ 0(s), 0) for x ∈ {σ (s)}×ޒ, s ∈ ޒ. With 00 the aid of e3 := (0, 0, 1) we deﬁne n(x) := t(x)∧e3 and κ(x) := − (σ (s), 0), n(x) for x ∈ {σ (s)} × ޒ, s ∈ ޒ. We can assume that n(x) points to the exterior of G. Obviously, t(x), e3 ∈ Tx ᏿, n(x) ⊥ Tx ᏿ for all x ∈ ᏿, where Tx ᏿ denotes the tangent space of ᏿ at x. Let B := {w = (u, v) ∈ ޒ2 : u2 + v2 < 1, v > 0} denote the semidisc. The boundary ∂ B consists of the interval I := (−1, 1) × {0} and the closed semicircle C :=∂ B\I . In the sequel, we consider immersions X : B →ޒ3 of class C0(B, ޒ3)∩ C3(B \ {−1, 1}, ޒ3) with their Gauss map

Xu ∧ X N : B \ {−1, 1} → ޒ3, N(w) := v |Xu ∧ Xv| 412 FRANK MÜLLER AND SVENWINKLMANN possessing finite area Z Ꮽ(X) = d A < ∞. B Here d A = |Xu ∧ Xv| du dv denotes the surface element with respect to the induced metric g. In order to extend the projectability and uniqueness result of [Hildebrandt and Sauvigny 1995] to parametric functionals, we introduce the following class Ꮿ(0, ᏿) of immersions: Definition 2.3. An immersion X = X(u, v) ∈ C0(B, ޒ3) ∩ C3(B \ {−1, 1}, ޒ3) with finite area Ꮽ(X) < ∞ is called admissible, and we write X ∈ Ꮿ(0, ᏿), if

(a) X|C : C → 0 maps C topologically onto 0 and X (−1, 0) = P1, X (1, 0) = P2, and (b) X (I ) ⊂ ᏿. Later we will need the following regularity assumptions, which allow us to control the curvature of an F-stationary immersion X ∈ Ꮿ(0, ᏿) at the corners w = ±1. Condition (R). The total curvature of X is bounded, i.e., Z (2-1) |K | d A < ∞, B and the limits (2-2) N(±1) := lim N(w) w→±1 exist. Remark. For stationary minimal surfaces, i.e. the case F(z) = |z|, one can show that both conditions (2-1), (2-2) are satisﬁed. In fact, this follows from asymptotic expansions at the corners w = ±1, see [Dierkes et al. 1992, Section 8.4]. Thus Condition (R) seems geometrically reasonable. For X ∈ Ꮿ(0, ᏿) we now consider the parametric functional Z Ᏺ(X) = F(N) d A B with a Lagrangian F ∈ C3(ޒ3 \{0}, ޒ) ∩ C0(ޒ3, ޒ) satisfying the homogeneity relation (1-1). Throughout this paper, F is assumed to be positive: F(z) > 0 for all z 6= 0. In addition, we always assume F to be elliptic; that is, the restriction of ∂2 F F z = z zz( ) α β ( ) ∂z ∂z α,β=1,2,3 F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES 413 to z⊥ := {ζ ∈ ޒ3 : hζ, zi = 0} is a positive-deﬁnite linear mapping for all z 6= 0. Geometrically, the ellipticity condition implies that F represents a support function of the convex body \ y ∈ ޒ3 : hy, zi ≤ F(z) . z6=0

Its boundary ᐃF gives us the convex surface parametrized by

2 3 (2-3) Fz : S → ޒ , z 7→ Fz(z).

2 In the terminology of [Taylor 1978], ᐃF = Fz(S ) is called the Wulff shape. 3 Given X ∈ Ꮿ(0, ᏿), we say that a smooth family X : B × (−ε0, ε0) → ޒ of immersions is an admissible variation of X if we have X( · , 0) = X,

∂ 2 3 Y := X( · , ε)| = ∈ C (B ∪ I, ޒ ), ∂ε ε 0 0

X(w, ε) = X(w) for all ε ∈ (−ε0, ε0) and all w ∈ (B ∪ I ) \ K with some compact set K ⊂ B ∪ I , and X( · , ε)|I : I → ᏿ for all ε ∈ (−ε0, ε0). Y is called the corresponding variational vector ﬁeld. Evidently, we deduce

(2-4) Y (w) ∈ TX(w)᏿ for all y ∈ I. ∈ 2 ∪ 3 Conversely, if Y C0 (B I, ޒ ) satisﬁes (2-4), one can show that an admissible variation of the form X( · , ε) = X + εY + o(ε) exists; see [Dierkes et al. 1992, Volume I, p. 333], for example. We say that X ∈ Ꮿ(0, ᏿) is F-stationary if the ﬁrst variation

d δᏲ(X, Y ) := Ᏺ X( · , ε) dε ε=0 vanishes for all admissible variations. An F-stationary immersion X ∈ Ꮿ(0, ᏿) is called F-stable if additionally the second variation

2 2 d δ Ᏺ(X, Y ) = Ᏺ X( · , ε) dε2 ε=0 is nonnegative for all admissible variations. Obviously, any minimizer X ∈ Ꮿ(0, ᏿) of Ᏺ is F-stable, but the converse is not true in general. Standard computations (see [Clarenz 2002, Section 1] or [Winklmann 2002, Proposition 2.1], for example) show that the ﬁrst variation for X ∈ Ꮿ(0, ᏿) is Z Z

(2-5) δᏲ(X, Y ) = − HF hY, Ni d A − Fz(N), Xu ∧ Y du. B I 414 FRANK MÜLLER AND SVENWINKLMANN

Here HF denotes the F-mean curvature or anisotropic mean curvature of X, de- ﬁned as follows [Rawer¨ 1993; Clarenz 2002]: Let

(2-6) NF := Fz ◦ N, NF : B \ {−1, 1} → ᐃF , describe the generalized Gauss map of X into the Wulff shape. Then SF := −1 −d X ◦ d NF is called the F-Weingarten operator and

(2-7) HF := tr SF . For technical reasons, we write

(2-8) SF := AF ◦ S, −1 where S := −d X ◦ d N denotes the classical Weingarten operator and AF indicates the symmetric, positive-deﬁnite endomorphism given by −1 (2-9) AF := d X ◦ Fzz(N) ◦ d X.

Note that AF is the identity if F(z) = |z| is the area integrand. Hence, in this case all deﬁnitions coincide with the classical notions. Now assume that X ∈ Ꮿ(0, ᏿) is F-stationary. If we choose Y = λN with ∈ 2 λ C0 (B, ޒ), we infer the identity

(2-10) HF ≡ 0 on B from (2-5) and the fundamental lemma of the calculus of variations. As a consequence, Z ∧ = ∈ 2 ∪ 3 Fz(N), Xu Y du 0 for all Y C0 (B I, ޒ ) satisfying (2-4). I This implies

(2-11) Fz(N(w)) ∈ TX (w)᏿ for all w ∈ I. Hence we have the following characterization of F-stationary immersions: Lemma 2.4. Let F be an elliptic Lagrangian and let {0, ᏿} be a projectable boundary configuration. X ∈ Ꮿ(0, ᏿) is F-stationary if and only if X satisfies (2-10) and the contact condition (2-11). We now derive two general relations, which represent the anisotropic analogues to the well known relations Nu = N ∧ Nv, Nv = −N ∧ Nu for conformally parametrized minimal surfaces. These will be particularly important in the deriva- tion of the boundary condition for the normal (Lemma 3.3). We will use standard shorthands when computing in coordinates, writing in- 1 2 differently (u, v) = (u , u ) and ϕu = ϕu1 = ϕ,1, ϕv = ϕu2 = ϕ,2. We denote the coefficients of the induced metric by gαβ = hX,α, X,β i, and the coefficients F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES 415

−1 αβ := = − 2 of (gαβ )α,β=1,2 by g . Moreover, we abbreviate g det(gαβ ) g11g22 g12 with a slight notational overlap. Finally, we let hαβ := g(S∂α, ∂β ) = −hN,α, X,β i indicate the coefﬁcients of the second fundamental form and aαβ = g(AF ∂α, ∂β ) = hFzz(N)X,α, X,β i the coefﬁcients of g(AF · , · ). As an immediate consequence of (2-7), (2-8), and (2-9) we obtain

αβ γ δ (2-12) SF ∂ε = g aβγ g hδε∂α, αβ γ δ (2-13) HF = g aβγ g hδα, where the Einstein summation convention is in effect. We also need the well known identities

√ 2α √ 1α (2-14) Xu ∧ N = − gg X,α, Xv ∧ N = gg X,α on B \ {−1, 1}, valid for an arbitrary immersion X ∈ Ꮿ(0, ᏿). Lemma 2.5. For any F-stationary immersion X ∈ Ꮿ(0, ᏿) we have ∂ √ (2-15) F (N) = gg2αa gβγ N ∧ N , ∂u z αβ ,γ ∂ √ (2-16) F (N) = − gg1αa gβγ N ∧ N ∂v z αβ ,γ on B \ {−1, 1}. Proof. We prove only the ﬁrst equality; the argument for the second is similar. First note that both sides of (2-15) are tangential to X; more precisely, ∂ √ F (N) = U α X , gg2αa gβγ N ∧ N = V α X . ∂u z ,α αβ ,γ ,α We will show that the coefﬁcients coincide, i.e., U α = V α for α = 1, 2. To this end we use (2-12) obtaining

α αβ γ δ (2-17) U = −g aβγ g hδ1. In order to compute V α, we employ (2-14) and deduce 1 1 g1β N ∧ N , X = √ h , g2β N ∧ N , X = −√ h . ,α ,β g 2α ,α ,β g 1α Consequently, we arrive at

1 2β γ δ 2 2β γ δ (2-18) V = g aβγ g hδ2, V = −g aβγ g hδ1. Comparison of (2-17) and (2-18) immediately yields U 2 = V 2. Furthermore, we 1 1 αβ γ δ see that V −U = g aβγ g hδα = HF , due to (2-13). Because X is supposed to 1 1 be F-stationary, we infer that U = V . 416 FRANK MÜLLER AND SVENWINKLMANN

2 α Let ϕ ∈ C (B) and a smooth vector ﬁeld V = V ∂α on B be prescribed. We introduce 1 √ ∇ϕ = gαβ ϕ ∂ and div V = √ ( gV α) = V α − 0α V β , ,α β g ,α ,α αβ

γ the gradient and divergence with respect to g; as usual, the 0αβ here are the Christoffel symbols, given by the Gauss–Weingarten relations (see [Dierkes et al. 1992, Chapter 1]) γ X,αβ = 0αβ X,γ + hαβ N.

Following [Clarenz 2002], we deﬁne the elliptic operator 1F of second order by 1 √ (2-19) 1 ϕ := div(A ∇ϕ) = √ ggαβa gγ δϕ . F F g βγ ,δ ,α

We recall the divergence of AF , given by the 1-form

α βγ (2-20) (div AF )[V ] := V g aαβ;γ ,

δ δ where aαβ;γ = aαβ,γ − 0γ αaδβ − 0γβaαδ denote the coefficients for the covariant derivative of the tensor g(AF · , · ). The following two identities were established in [Clarenz 2002, Theorem 2; Clarenz and von der Mosel 2004, Corollary 4.3]. Using (2-19) and (2-20), the first of them is derived via the Gauss–Weingarten relations, and the second identity via the Codazzi equation hαβ;γ = hβγ ;α. Lemma 2.6. Let F be an elliptic Lagrangian. Then any F-stationary immersion X ∈ Ꮿ(0, ᏿) fulfills the equations

(2-21) 1F X − (divAF )[∇ X] = 0,

2 (2-22) 1F N + tr(AF S )N = 0 on B \ {−1, 1}. We conclude this section with a general assumption on F, which has two consequences: it forces any F-stationary surface X ∈ Ꮿ(0, ᏿) in an admissible boundary conﬁguration {0, ᏿} to map I onto 6 ×ޒ (Lemma 3.1), and it ensures that N 3 > 0 at the corners (Lemma 3.2).

Condition (W). The Wulff shape ᐃF meets E perpendicularly, and ᐃF ∩ E = ∂ BR(0) ∩ E for some radius R > 0. According to (2-3), this condition is equivalent to

1 2 1 2 1 (2-23) Fz(z , z , 0) = (Rz , Rz , 0) for all z ∈ S × {0}. F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES 417

3. Boundary behaviour of F-stationary immersions

Lemma 3.1. Let F denote an elliptic Lagrangian satisfying Condition (W), and let {0, ᏿} represent an admissible boundary conﬁguration. Suppose X ∈ Ꮿ(0, ᏿) to be F-stationary. Deﬁning f (w) = (X 1(w), X 2(w)) : B → E, we then infer f (I ) = 6. Proof. We follow the proof of [Hildebrandt and Sauvigny 1992, Proposition 1]. ∗ ∗ Let s ∈ (−∞, s1] be the largest number such that f (B) ⊂ H(s ), where we have made the abbreviation 0 H(s) := y ∈ E : y − σ (s), σ (s) ≥ 0 , s ∈ (−∞, s1].

Because f : B → E is continuous, such an s∗ exists by (a) in Definition 2.1. ∗ Suppose that s < s1. Then the nonnegative function 8(w) := f (w) − σ (s∗), σ 0(s∗) , w ∈ B, satisfies the homogeneous elliptic equation 1F 8 − (div AF )[∇8] = 0 on B, by Lemma 2.6. According to the maximum principle and the choice of s∗, we can find a point w0 ∈ ∂ B \ {−1, 1} with 8(w0) = 0. From condition (b) in Definition ∗ 2.2 and the boundary conditions for X we infer that w0 ∈ I and f (w0) = σ (s ). Hopf’s boundary point lemma now implies 8u(w0) = 0 and 8v(w0) > 0, which we may rewrite as

(3-1) Xu(w0), t(X (w0)) = 0, Xv(w0), t(X(w0)) > 0. ∈ = 3 Noting that Xu(w0) TX (w0)᏿ we ﬁnd Xu(w0) Xu(w0)e3. This reveals that 3 N (w0) = N(w0), e3 = 0, and (2-11) together with Condition (W) imply N(w0) =

N(w0), t(X(w0)) t(X (w0)). With the aid of (2-14) and (3-1), we now obtain the contradiction

√ 2α 0 > − gg X,α(w0), t(X(w0)) = Xu(w0) ∧ N(w0), t(X(w0)) = 0.

∗ Thus we conclude s = s1 and hence f (B) ⊂ H(s1). Similarly, one shows that f (B) ⊂ H(s2) holds true with 0 H(s) := y ∈ E : y − σ (s), σ (s) ≤ 0 , s ∈ [s2, +∞). A further application of Hopf’s boundary point lemma finally yields f (I ) = 6. Form Lemma 3.1 we infer that f (∂ B) = ∂G. A standard argument then proves transversality to the fixed boundary C. Lemma 3.2. In addition to the assumptions of Lemma 3.1, suppose that X ∈ Ꮿ(0, ᏿) satisfies Condition (R). Then N 3(w) > 0 for all w ∈ C. 418 FRANK MÜLLER AND SVENWINKLMANN

Proof. By [Clarenz 2002, Theorem 2.3], F-stationary immersions have the convex- hull property. Hence, the argument of [Sauvigny 1982, Satz 2] applies and yields the estimate N 3(w) > 0 for arbitrary w ∈ C \{−1, 1}. See also [Clarenz and von der Mosel 2004, p. 33]. To prove that N 3(w) > 0 for all w ∈ C, suppose that N 3(−1) vanished. Then − ∈ Condition (W), (2-11) and the continuity of N would imply N( 1) TP1 ᏿; hence

(3-2) N(−1) = hN(−1), t(P1)it(P1). 3 On the other hand, we have hN(−1), a(P1)i = 0, where a(P1) ∈ ޒ denotes a unit tangent vector to 0 in P1. Combining this with (3-2), we infer the relation

ht(P1), a(P1)i = 0.

However, because {0, ᏿} is projectable, this is only possible if 0 meets 60 perpendicularly, in contradiction to condition (a) in Deﬁnition 2.2. Thus we must have N 3(−1) 6= 0, and by continuity even N 3(−1) > 0. The same argument applies to 3 N (+1) and the proof is complete. Now we derive a boundary condition for N 3 on I which generalizes [Hildebrandt and Sauvigny 1995, Proposition 1]. Lemma 3.3. Let F be an elliptic Lagrangian, and let X ∈ Ꮿ(0, ᏿) be an F- stationary immersion in a projectable boundary conﬁguration {0, ᏿}. Writing F = F(N), κ = κ(X), etc., we have

√ 2α βγ −1 3 −3 2 3 (3-3) gg aαβ g [F N ],γ = F κ Fz, t Fz ∧ Xu, n N on I. Proof. First note the relation −1 3 −1 −1 F[F N ],γ = F (F N),γ , e3 = −F hFz, N,γ ihN, e3i + hN,γ , e3i.

Because hFz(N), Ni = F(N) (by homogeneity), this implies −1 3 −2 [F N ],γ = F hN ∧ N,γ , Fz ∧ e3i on B \ {−1, 1}. In view of Lemma 2.5, we arrive at

√ − − D ∂ E − D ∂ E (3-4) gg2αa gβγ [F 1 N 3] = F 2 F , F ∧e = −F 2 F , [F ∧e ] αβ ,γ ∂u z z 3 z ∂u z 3 on B \ {−1, 1}. From (2-11) we conclude Fz ∧e3 = Fz ∧(n∧t) = hFz, tin on I and consequently D ∂ E D ∂ E 2 (3-5) F , [F ∧ e ] = F , t F , n = κ F , t X , t on I. z ∂u z 3 z z ∂u z u In the last identity, we used the general relation

(3-6) v, Dn(x)w = κ(x) v, t(x) w, t(x) for all v, w ∈ Tx ᏿, x ∈ ᏿, F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES 419 due to the cylindrical structure of ᏿. On the other hand, we compute

hXu, tiFz − hFz, tiXu = (Xu ∧ Fz) ∧ (e3 ∧ n) = hXu ∧ Fz, nie3 on I, by applying (2-11) and t = e3 ∧ n. By multiplication with N we ﬁnally deduce 3 (3-7) FhXu, ti = −hFz ∧ Xu, niN on I.

Now the relation (3-3) stated above results from (3-4), (3-5), and (3-7).

4. The second variation

In this section we consider the second variation of an F-stationary immersion X ∈ Ꮿ(0, ᏿) in a projectable boundary configuration. For second variation formulas under Plateau type boundary conditions we refer the reader to [Rawer¨ 1993], [Frohlich¨ 2002] and [Clarenz and von der Mosel 2004]. Let X be an admissible variation of X ∈ Ꮿ(0, ᏿) with the variational vector ∈ 2 ∪ 3 field Y C0 (B I, ޒ ). We denote by N(ε), d A(ε), HF (ε) and Y (ε) geometric quantities evaluated at X( · , ε). Differentiating the first variation formula (2-5), we obtain

δ2Ᏺ(X,Y )

d = δᏲ(X(·,ε),Y (ε)) dε ε=0 Z Z d = − HF (ε)hY (ε), N(ε)id A(ε) − hFz(N(ε)), X u(·,ε) ∧ Y (ε)idu . dε B I ε=0 We now assume that X is F-stationary. Then we infer from (2-10) and (2-11) that Z 2 = − ∂ h i (4-1) δ Ᏺ(X, Y ) HF (ε) Y, N d A B ∂ε ε=0 Z D E − ∂ ∧ Fz(N(ε)) , Xu Y du I ∂ε ε=0 Z D E − ∂ · ∧ Fz(N), X u( , ε) Y (ε) du. I ∂ε ε=0 According to [2004, Section 4], the variation of the F-mean curvature is

∂ 2 HF (ε) = 1F ϕ + ϕ tr(AF S ) ∂ε ε=0 where ϕ = hY, Ni. Integration by parts consequently yields Z Z ∂ 2 2 (4-2) − HF (ε) hY, Ni d A = g(AF ∇ϕ, ∇ϕ) − tr(AF S )ϕ d A ∂ε ε=0 B B Z √ 2α βγ + ϕ gg aαβ g ϕ,γ du. I 420 FRANK MÜLLER AND SVENWINKLMANN

Furthermore, we note the relation

∂ αβ αβ N(ε) = −g hY,β , NiX,α = g hY, N,β i − ϕ,β X,α. ∂ε ε=0

In view of (2-14) and the identity Fzz(N)N = 0, this gives us

D ∂ E √ 2α βγ √ 2α βγ (4-3) Fz(N(ε)) , Xu∧Y =ϕ gg aαβ g ϕ,γ −ϕ gg aαβ g Y, N,γ . ∂ε ε=0

Note that Fzz(N)N = 0 is an immediate consequence of the homogeneity relation (1-1). Finally, we observe that

X u( · , ε) ∧ Y (ε) = X u( · , ε) ∧ Y (ε), n(X( · , ε)) n(X( · , ε)) on I, because Y (ε) is always tangential to ᏿. Using Fz(N), n = 0 and (3-6), we thus obtain D ∂ E (4-4) Fz(N), X u( · , ε) ∧ Y (ε) = κ Xu ∧ Y, n Fz(N), t Y, t . ∂ε ε=0 Collecting formulas (4-1)–(4-4), we arrive at Z 2 n 2 2o δ Ᏺ(X, Y ) = g(AF ∇ϕ, ∇ϕ) − tr(AF S )ϕ d A B Z Z √ 2α βγ + ϕ gg aαβ g Y, N,γ du − κ Xu ∧ Y, n Fz(N), t Y, t du, I I with ϕ = Y, N . −1 Variational vector ﬁelds of the form Y = λF(N) Fz(N) with some function ∈ 2 ∪ = = λ C0 (B I, ޒ) are of special interest. According to Y, N λ and Y, N,γ −1 λF(N) F(N),γ , we obtain:

Theorem 4.1. Let F be an elliptic Lagrangian and let X ∈ Ꮿ(0, ᏿) be an F- stationary immersion in a projectable boundary conﬁguration {0, ᏿}. For any λ ∈ 2 ∪ = −1 C0 (B I, ޒ) the second variation of X in the direction Y λF(N) Fz(N) is then given by Z 2 2 2 (4-5) δ Ᏺ(X, λ) = g(AF ∇λ, ∇λ) − tr(AF S )λ d A B Z 2√ −1 2α βγ + λ gF(N) g aαβ g F(N),γ du I Z 2 −2 2 − λ κ F(N) Xu ∧ Fz(N), n(X) Fz(N), t(X) du. I F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES 421

2 2 −1 Here we have set δ Ᏺ(X, λ) := δ Ᏺ(X, λF(N) Fz(N)). In particular, for all F-stable X we have

(4-6) δ2Ᏺ(X, λ) ≥ 0 ∈ 2 ∪ with arbitrary λ C0 (B I, ޒ). ∈ 1 ∩ 0 ∪ Remark. Observe that (4-6) remains true for λ H2 (B, X) C0 (B I ), where we have set Z 1 2 2 H2 (B, X) = λ : B → ޒ measurable : {λ + |∇λ| }d A < +∞ . B Using the essential assumption (W) and the regularity hypothesis (R), we now show that (N 3)− := min{N 3, 0} is an admissible test function. Lemma 4.2. Let F be an elliptic Lagrangian satisfying the Condition (W), and let {0, ᏿} be an admissible boundary conﬁguration. Suppose furthermore that X ∈ Ꮿ(0, ᏿) denotes an F-stationary immersion satisfying Condition (R). Then 3 − 1 ∩ 0 ∪ (N ) lies in H2 (B, X) C0 (B I ). 3 − ∈ 0 ∪ Proof. Clearly, (N ) C0 (B I ) in view of the continuity assumption (2-2) and Lemma 3.2. 3 − ∈ 1 In order to prove (N ) H2 (B, X), we utilize (2-1) and argue as follows: Fix w ∈ B, and let {e1, e2} be an orthonormal basis of Tw B such that Sei = κi ei for i = 1, 2 hold true at this point; here κ1, κ2 denote the principal curvatures of X. Because HF vanishes, we have the relation

(4-7) α1κ1 + α2κ2 = 0 − + with αi := g(AF ei , ei ). Now we estimate 0 < 3 ≤ α1, α2 ≤ 3 < ∞ where hF (z)ζ, ζi hF (z)ζ, ζi 3− := zz , 3+ := zz (4-8) inf 2 sup 2 z∈S2 |ζ | ∈ 2 |ζ| ⊥ z S ζ∈z \{0} ζ∈z⊥\{0} give a lower and upper bound for the eigenvalues of AF , respectively. A combination with (4-7) yields the estimate 3+ κ2 + κ2 ≤ 2 |K | 1 2 3− = |∇ |2 = 2 + 2 where K κ1κ2 denotes the Gaussian curvature of X. Due to N κ1 κ2 , we conclude that Z + Z 2 3 |∇ N| d A ≤ 2 − |K | d A. B 3 B 1 Thus N lies in H2 (B, X) and the assertion follows. 422 FRANK MÜLLER AND SVENWINKLMANN

Remark. Sauvigny [1990, Lemma 7] has used similar arguments in order to establish curvature estimates for immersions of minimal-surface type in weighted conformal parameters.

5. Projectability and uniqueness

In this section we prove our main results.

Theorem 5.1. Let F denote an elliptic Lagrangian satisfying Condition (W), and let {0, ᏿} constitute an admissible boundary conﬁguration. Furthermore, let X ∈ Ꮿ(0, ᏿) describe an F-stable immersion satisfying Condition (R). Then we have

(5-1) N 3(w) > 0 on B and X can be represented as a graph over the x1, x2-plane, i.e., we have the parametrization x3 = ζ(x1, x2), (x1, x2) ∈ G, with some function

3 0 ζ ∈ C (G \{p1, p2}) ∩ C (G), where p1 := σ (s1) and p2 := σ (s2).

Proof. According to Lemma 4.2 and the preceding remark, we know that the func- − := { } ∈ 1 ∩ 0 ∪ := 3 tion ω min ω, 0 H2 (B, X) C0 (B I ) with ω N is admissible in the second variation formula (4-5). Using (2-22), an integration by parts yields Z 2 − −√ 2α βγ 3 δ Ᏺ(X, ω ) = − ω gg aαβ g N,γ du I Z − 3√ −1 2α βγ + ω N gF(N) g aαβ g F(N),γ du I Z − 3 −2 2 − ω N κ F(N) Xu ∧ Fz(N), n(X) Fz(N), t(X) du I Z −√ 2α βγ −1 3 = − ω gF(N)g aαβ g [F(N) N ],γ du I Z − 3 −2 2 − ω N κ F(N) Xu ∧ Fz(N), n(X) Fz(N), t(X) du. I

Lemma 3.3 then reveals that δ2Ᏺ(X, ω−) = 0. From here onwards we can argue as in [Hildebrandt and Sauvigny 1995]: Deﬁn- := 2 − + ∈ ∞ ing 8(ε) δ Ᏺ(X, ω εϕ) with arbitrary ϕ C0 (B), the stability inequality (4-6) implies 80(0) = 0. This is equivalent to Z − 2 − ∞ g(AF ∇ω , ∇ϕ) − tr(AF S )ω ϕ d A = 0 for all ϕ ∈ C0 (B). B F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES 423

− 0 ∪ − ≡ Because ω lies in C0 (B I ), Moser’s weak Harnack inequality yields ω 0, i.e., N 3 ≥ 0 on B. This gives us N 3 > 0 on B ∪ C, which is a consequence of (2-22) and N 3 > 0 near C in conjunction with Harnack’s inequality. 3 3 Finally, we establish that N > 0 on I . Indeed, if N (w0) = 0 were true for some w0 ∈ I , Lemma 3.3 would imply

2α βγ 3 (5-2) g aαβ g N,γ (w0) = 0.

3 3 On the other hand, Hopf’s boundary point lemma yields Nu (w0) = 0 and Nv (w0) 6= δα βγ 2α β2 0. According to the deﬁniteness of the matrix (g aαβ g ) we have g aαβ g 6= 0, and (5-2) generates a contradiction proving (5-1). Due to Lemma 3.1, we have f |∂ B : ∂ B → ∂G topologically. Indeed, we already know f |C : C → 0 topologically by assumption, and (5-1) yields J f (w) > 0 on B \ {−1, 1}, thus | fu| > 0 on I as well. In addition, an index argument yields f |B : B → G topologically. In fact, this follows from J f (w) > 0 on B, the boundary behaviour of f and the well known index-sum formula, compare [Sauvigny 2006, Chapter III]. Finally, the implicit function theorem reveals ζ(x1, x2) := X 3 ◦ f −1(x1, x2) ∈ 3 0 C (G \{p1, p2}) ∩ C (G). We conclude with a geometric uniqueness result. Theorem 5.2. Let F be an elliptic Lagrangian satisfying Condition (W), and let {0, ᏿} denote an admissible boundary conﬁguration. Then, apart from reparam- etrizations, there exists at most one F-stable immersion X ∈ Ꮿ(0, ᏿) satisfying Condition (R). Remark. Again we refer the reader to [Hildebrandt and Sauvigny 1995] and [Muller¨ 2005], concerning comparable results for stable minimal surfaces and surfaces of prescribed mean curvature, respectively. The existence of F-stationary immersions with partially free boundaries has not yet been proved, but see [Hilde- brandt and von der Mosel 2002] for related results. For the construction of an embedded F-minimal surface bounding a closed smooth Jordan curve 0 ⊂ ޒ3, see [White 1991]. Proof of Theorem 5.2. According to Theorem 5.1, any F-stable immersion X ∈ Ꮿ(0, ᏿) satisfying Condition (R) can be represented as a graph

1 2 3 −1 1 2 3 0 ζ(x , x ) = X ◦ f (x , x ) ∈ C (G \{p1, p2}) ∩ C (G). Moreover, this graph representation has the same orientation as X, due to (5-1). Because X is F-stationary, the height function ζ is a critical point of the nonpara- metric functional Z F[ζ] = f (Dζ ) dx, G 424 FRANK MÜLLER AND SVENWINKLMANN

3 2 where we have written f (q) = f (q1, q2) := F(−q1, −q2, 1) ∈ C (ޒ ) and Dζ = (ζx1 , ζx2 ). In particular, the function ζ solves the mixed boundary value problem ∂ ∂ f Q := D = G ζ α ( ζ ) 0 on , (5-3) ∂x ∂qα

ζ = γ on 0, h fq (Dζ ), νi = 0 on 6. Here γ = γ (x1, x2) denotes the given height function above 0, and ν = ν(x1, x2) is the normal of 6 which points to the exterior of G. In view of the ellipticity of F, we infer the minimal surface type condition ∂2 f 3− hq, ξi2 (5-4) (q)ξ αξ β ≥ |ξ|2 − p 2 ∂qα∂qβ 1 + |q|2 1 + |q| 1 2 2 − for all q = (q1, q2), ξ = (ξ , ξ ) ∈ ޒ , where 3 is the positive number given by (4-8); see also [Finn 1954; Simon 1977]. Now assume we had two F-stable immersions X1, X2 ∈ Ꮿ(0, ᏿) such that 3 −1 1 2 3 0 ζl = Xl ◦ fl (x , x ) ∈ C (G \{p1, p2}) ∩ C (G) with l = 1, 2 satisfy (5-3). Then the difference function ζ := ζ1 − ζ2 solves a linear elliptic equation on G 1 with their coefﬁcients in C (G \{p1, p2}), compare [Gilbarg and Trudinger 1983] and [Sauvigny 2006, Chapter VI, § 2]. According to the maximum principle, the function ζ has to assume its maximum and minimum on 6 ∪ 0. We now infer (5-5) M · Dζ, ν = 0 on 6 from the second boundary condition in (5-3), where we abbreviated Z 1 1 2 M = M(x , x ) := fqq t Dζ1 + (1 − t)Dζ2 dt. 0

If the function ζ assumed a positive maximum at the point x0 ∈ 6, Hopf’s boundary point lemma would imply Dζ(x0) = hDζ(x0), νiν with hDζ(x0), νi > 0. In view of (5-4), we would have

hM · Dζ(x0), νi = hDζ(x0), νihMν, νi > 0, contradicting (5-5). Similarly, one excludes a negative minimum on 6. Hence we infer ζ ≡ 0 on G and the announced result follows.

References

[Clarenz 2002] U. Clarenz, “Enclosure theorems for extremals of elliptic parametric functionals”, Calc. Var. Partial Diff. Eq. 15:3 (2002), 313–324. MR 2003m:49003 Zbl 1018.53006 [Clarenz and von der Mosel 2004] U. Clarenz and H. von der Mosel, “On surfaces of prescribed F-mean curvature”, Paciﬁc J. Math. 213:1 (2004), 15–36. MR 2005b:53010 Zbl 1058.53003 F-STABLE IMMERSIONS WITH PARTIALLY FREE BOUNDARIES 425

[Dierkes et al. 1992] U. Dierkes, S. Hildebrandt, A. Küster, and O. Wohlrab, Minimal surfaces (2 vol.), Grundlehren der Math. Wissenschaften 295–296, Springer, Berlin, 1992. MR 94c:49001b Zbl 0777.53012 [Finn 1954] R. Finn, “On equations of minimal surface type”, Ann. of Math. (2) 60 (1954), 397–416. MR 16,592b Zbl 0058.32501 [Fröhlich 2002] S. Fröhlich, “Curvature estimates for µ-stable G-minimal surfaces and theorems of Bernstein type”, Analysis (Munich) 22:2 (2002), 109–130. MR 2003f:35108 Zbl 01803957 [Gilbarg and Trudinger 1983] D. Gilbarg and N. S. Trudinger, Elliptic partial differential equations of second order, 2nd ed., Grundlehren der Math. Wissenschaften 224, Springer, Berlin, 1983. MR 86c:35035 Zbl 0562.35001 [Hildebrandt and Sauvigny 1991] S. Hildebrandt and F. Sauvigny, “Embeddedness and uniqueness of minimal surfaces solving a partially free boundary value problem”, J. Reine Angew. Math. 422 (1991), 69–89. MR 92k:58056 Zbl 0729.53014 [Hildebrandt and Sauvigny 1992] S. Hildebrandt and F. Sauvigny, “On one-to-one harmonic mappings and minimal surfaces”, Nachrichten Akad. Wiss. Göttingen Math.-Phys. Kl. II 3 (1992), 21. MR 94b:49069 Zbl 0794.35050 [Hildebrandt and Sauvigny 1995] S. Hildebrandt and F. Sauvigny, “Uniqueness of stable minimal surfaces with partially free boundaries”, J. Math. Soc. Japan 47:3 (1995), 423–440. MR 96a:58061 Zbl 0831.53006 [Hildebrandt and von der Mosel 1999] S. Hildebrandt and H. von der Mosel, “On two-dimensional parametric variational problems”, Calc. Var. Partial Diff. Eq. 9 (1999), 249–267. MR 2000h:49006 Zbl 0934.49022 [Hildebrandt and von der Mosel 2002] S. Hildebrandt and H. von der Mosel, “The partially free boundary problem for parametric double integrals”, pp. 145–165 in Nonlinear problems in mathematical physics and related topics, vol. I, edited by M. S. Birman et al., Kluwer/Plenum, New York, 2002. MR 2004b:49004 Zbl 1055.49024 [Hildebrandt and von der Mosel 2005] S. Hildebrandt and H. von der Mosel, “Conformal representation of surfaces, and Plateau’s problem for Cartan functionals”, Preprint 2005-07-05, RWTH, Aachen, 2005. To appear in Riv. Mat. Univ. Parma. [Kneser 1926] H. Kneser, “Lösung der Aufgabe 41”, Jahresber. Deutsch. Math.-Ver. 35 (1926), 123– 124. [Koiso and Palmer 2006] M. Koiso and B. Palmer, “Stability of anisotropic capillary surfaces between two parallel planes”, Calc. Var. Partial Diff. Eq. 25:3 (2006), 275–298. MR 2006j:58018 Zbl 05014072 [Müller 2005] F. Müller, “On stable surfaces of prescribed mean curvature with partially free boundaries”, Calc. Var. Partial Diff. Eq. 24:3 (2005), 289–308. MR 2007c:53013 Zbl 1079.53018 [Radó 1926] T. Radó, “Aufgabe 41”, Jahresber. Deutsch. Math.-Ver. 35 (1926), 49. [Räwer 1993] K. Räwer, Stabile Extremalen parametrischer Doppelintegrale im ޒ3, Dissertation, Universität Bonn, 1993. [Sauvigny 1982] F. Sauvigny, “Flächen vorgeschriebener mittlerer Krümmung mit eineindeutiger Projektion auf eine Ebene”, Math. Z. 180:1 (1982), 41–67. MR 83j:53004 Zbl 0465.53003 [Sauvigny 1990] F. Sauvigny, “Curvature estimates for immersions of minimal surface type via uniformization and theorems of Bernstein type”, Manuscripta Math. 67:1 (1990), 69–97. MR 91b: 53010 Zbl 0703.53050 426 FRANK MÜLLER AND SVENWINKLMANN

[Sauvigny 2006] F. Sauvigny, Partial differential equations, 1: Foundations and integral representations, Universitext, Springer, Berlin, 2006. Expanded translation of Partielle Differentialglei- chungen der Geometrie und der Physik: Grundlagen und Integraldarstellungen, Springer, Berlin, 2004. MR 2007c:35001 Zbl 1049.35001 [Simon 1977] L. Simon, “Equations of mean curvature type in 2 independent variables”, Paciﬁc J. Math. 69:1 (1977), 245–268. MR 56 #13099 Zbl 0354.35040 [Taylor 1978] J. E. Taylor, “Crystalline variational problems”, Bull. Amer. Math. Soc. 84:4 (1978), 568–588. MR 58 #12649 Zbl 0392.49022 [White 1991] B. White, “Existence of smooth embedded surfaces of prescribed genus that minimize parametric even elliptic functionals on 3-manifolds”, J. Differential Geom. 33:2 (1991), 413–443. MR 92e:58048 Zbl 0737.53009 [Winklmann 2002] S. Winklmann, “Isoperimetric inequalites involving generalized mean curvature”, Analysis (Munich) 22:4 (2002), 393–403. MR 2003j:53096 Zbl 1033.53052 [Winklmann 2003] S. Winklmann, “Existence and uniqueness of F-minimal surfaces”, Ann. Global Anal. Geom. 24:3 (2003), 269–277. MR 2004g:53015 Zbl 1027.53068

Received August 25, 2005.

FRANK MULLER¨ BRANDENBURGISCHE TECHNISCHE UNIVERSITAT¨ COTTBUS INSTITUTFUR¨ MATHEMATIK KONRAD-ZUSE-STRASSE 1 03044 COTTBUS GERMANY [email protected]

SVEN WINKLMANN UNIVERSITAT¨ DUISBURG-ESSEN CAMPUS DUISBURG FACHBEREICH MATHEMATIK 47048 DUISBURG GERMANY [email protected] http://www.uni-duisburg.de/FB11/DGL/Winklmann/wink PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC

HANS SCHOUTENS

In this paper, various Homological Conjectures are studied for local rings which are locally finitely generated over a discrete valuation ring V of mixed characteristic. Typically, we can only conclude that a particular conjecture holds for such a ring provided the residual characteristic of V is sufficiently large in terms of the complexity of the data, where the complexity is primar- ily given in terms of the degrees of the polynomials over V that define the data, but possibly also by some additional invariants such as (homological) multiplicity. Thus asymptotic versions of the Improved New Intersection Theorem, the Monomial Conjecture, the Direct Summand Conjecture, the Hochster–Roberts Theorem and the Vanishing of Maps of Tors Conjecture are given. That the results only hold asymptotically is due to the fact that nonstandard arguments are used, relying on the Ax–Kochen–Ershov Principle, to infer their validity from their positive characteristic counterparts. A key role in this transfer is played by the Hochster–Huneke canonical construction of big Cohen–Macaulay algebras in positive characteristic via absolute integral closures.

1. Introduction

In the last three decades, all the so-called Homological Conjectures have been settled completely for noetherian local rings containing a field by work of Peskine and Szpiro [1973], Hochster and Roberts [1974], Hochster [1975b; 1983], Evans and Griffith [1981] and others, to cite just some of the key papers. More recently, Hochster and Huneke have given more simplified proofs of most of these results by means of their tight closure theory, including their canonical construction of big Cohen–Macaulay algebras in positive characteristic (see [Hochster and Huneke 1992; 1989; 2000; Huneke 1996]; for further discussion and proofs, see [Bruns and Herzog 1993, §9] or [Strooker 1990]).

MSC2000: 13D22, 13L05, 03H05. Keywords: Homological conjectures, mixed characteristic, big Cohen–Macaulay algebra, Ax–Kochen–Ershov, improved new intersection theorem, vanishing of maps of tors. Partially supported by a grant from the National Science Foundation.

427 428 HANS SCHOUTENS

In sharp contrast is the development in mixed characteristic, where only spo- radic results (often in low dimensions) are known, apart from the breakthrough by Roberts [1987], settling the New Intersection Theorem for all noetherian local rings, and the recent work of Heitmann [2002] in dimension three. Some attempts have been made by Hochster, either by finding a suitable substitute for tight closure in mixed characteristic [1994], or by constructing big Cohen–Macaulay modules in mixed characteristic [1975a]. These approaches have yet to bear fruit and the best result to date in this direction is the existence of big Cohen–Macaulay algebras in dimension three [Hochster 2002], which in turn relies on the positive solution of the Direct Summand Conjecture in dimension three by Heitmann [2002]. In this paper, we will follow the big Cohen–Macaulay algebra approach, but instead of trying to work with rings of Witt vectors, we will use the Ax–Kochen– Ershov Principle [Ax and Kochen 1965; Ershov 1965; 1966], linking complete discrete valuation rings in mixed characteristic with complete discrete valuation rings in positive characteristic via an equicharacteristic zero (nondiscrete) valuation ring (see Theorem 2.3 below). This intermediate valuation ring is obtained by a construction which originates from logic, but is quite algebraic in nature, to wit, the ultraproduct construction. Roughly speaking, this construction associates to an infinite collection of rings Cw their ultraproduct C∞, which should be thought of as a kind of “limit” or “average” (realized as a certain homomorphic image of the product). An ultraproduct inherits many of the algebraic properties of its components. The correct formulation of this transfer principle is Łos’ Theorem, which makes precise when a property carries over (namely, when it is first order definable in some suitable language). Properties that carry over are those of being a domain, a field, a valuation ring, local, henselian; among the properties that do not carry over is noetherianness, so that almost no ultraproduct is noetherian (except an ultraproduct of fields or of artinian rings of bounded length). This powerful tool is used in [Schmidt-Gottsch¨ 1987; van den Dries and Schmidt 1984; Schoutens 2000a; 2007], to obtain uniform bounds in polynomial rings over fields; in [Schoutens 2000a; 2000b; 2003a; 2003c], to transfer properties from positive to zero characteristic; and in [Aschenbrenner and Schoutens 2007; Schoutens 2003d; 2004a; 2005a; 2005b], to give an alternative treatment of tight closure theory in equicharacteristic zero. The key fact in the first set of papers is a certain flatness result about ultraproducts (see Theorem 2.2 below for a precise formulation), and in the two last sets, the so-called Lefschetz Principle for algebraically closed fields (the ultraproduct of the algebraic closures of the p-element fields ކp is isomorphic to ރ). The Ax–Kochen–Ershov Principle is a kind of Lefschetz Principle for henselian valued fields, and its most concrete form states that the ultraproduct of all ކp t , with t a single indeterminate, is isomorphic to the ultraproduct of all rings of p-adicJ K ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 429 integers ޚp. We will identify both ultraproducts and denote the resulting ring by O. It follows that O is an equicharacteristic zero henselian valuation ring with principal maximal ideal, whose separated quotient (i.e., reduction modulo the intersection of all powers of the maximal ideal) is an equicharacteristic zero excellent complete discrete valuation ring.

Z-affine algebras. To explain the underlying idea in this paper, we introduce some notation. Let (Z, p) be a (not necessarily noetherian) local ring. A Z-affine algebra C is any Z-algebra of the form C = Z[X]/I where X is a finite tuple of indeterminates and I a finitely generated ideal in Z[X].A local Z-affine algebra is any localization R = Cm of a Z-affine algebra C with respect to a prime ideal m of C lying above p. In particular, the natural homomorphism Z → R is local. We denote the category of all local Z-affine algebras by Aff(Z). The objective is to transfer algebraic properties (such as the homological Conjec- tures) from the positive characteristic categories Aff(ކp t ) to the mixed character- J K istic categories Aff(ޚp). This will be achieved through the intermediate equicharacteristic zero category Aff(O). As this latter category consists mainly of nonnoetherian rings, we will have to find analogues in this setting of many familiar notions from commutative algebra, such as dimension, depth, Cohen–Macaulayness or regularity (see Sections 5 and 6). The following example is paradigmatic: let X be a finite tuple of indeterminates eq [ ] mix and let LO (A) be the ultraproduct of all ކp t X , and LO (A), the ultraproduct J K of all ޚp[X]. Note that both rings contain O, and in fact, contain O[X]. The key algebraic fact, which is equivalent to a result on effective bounds by Aschenbrenner [ ] ⊆ eq [ ] ⊆ mix [2001a], is that both inclusions O X LO (A) and O X LO (A) are flat. Sup- pose we have in each ކp t [X] a polynomial f p, and let f ∞ be their ultraproduct. J K ∈ eq A priori, we can only say that f ∞ LO (A). However, if all f p have X-degree d, for some d independent from p, then f ∞ itself is a polynomial over O of degree d (since an ultraproduct commutes with finite sums by Łos’ Theorem). Hence, as [ ] mix f ∞ lies in O X , we can also view it as an element in LO (A). Therefore, there are ˜ ˜ polynomials f p ∈ ޚp[X] whose ultraproduct is equal to f ∞. The choice of the f p is not unique, but any two choices will be equal for almost all p, by Łos’ Theorem. In conclusion, to a collection of polynomials defined over the various ކp t , of uniformly bounded degree, we can associate, albeit not uniquely, a collectionJ K of polynomials defined over the various ޚp (of uniformly bounded degree), and of course, this also works the other way. Instead of doing this for just one polynomial in each component, we can now do this for a finite tuple of polynomials of fixed length. If at the same time, we can maintain certain algebraic relations among them (characterizing one of the properties we seek to transfer), we will have achieved our goal. 430 HANS SCHOUTENS

Unfortunately, it is the nature of an ultraproduct that it only captures the “average” property of its components. In the present context, this means that the desired property does not necessarily hold in all ޚp[X], but only in almost all. In conclusion, we cannot hope for a full solution of the Homological Conjectures by this method, but only an asymptotic solution. In view of the above, the following deﬁnition is now natural.

Complexity. Let C be a Z-affine algebra, say, of the form C = Z[X]/I , with X a finite tuple of indeterminates and I a finitely generated ideal, and let R = Cm be a local Z-affine algebra (so that p ⊆ m). We say that C has Z-complexity at most c, if |X| ≤ c and I is generated by polynomials of degree at most c; we say that R has Z-complexity at most c, if, moreover, also m is generated by polynomials of degree at most c. An element r ∈ C is said to have Z-complexity at most c, if C has Z-complexity at most c and r is the image of a polynomial in Z[X] of degree at most c. An element r ∈ R has Z-complexity at most c, if R has Z-complexity at most c and if r is (the image of) a quotient P/Q of polynomials of degree at most c with Q ∈/ m. We say that a tuple or a matrix has Z-complexity at most c, if each of its entries has Z-complexity at most c and the number of entries is also bounded by c. Note that in a Z-affine algebra, the sum of two elements of Z-complexity at most c, has again Z-complexity at most c, whereas in a local Z-affine algebra, the sum has Z-complexity at most 2c. An ideal J in C or R has Z-complexity at most c, if it is generated by a tuple of Z-complexity at most c.A Z-algebra homomorphism C → C0 or a local Z- algebra homomorphism R → R0 is said to have Z-complexity at most c, if C and C0 (respectively, R and R0) are (local) Z-affine algebras of Z-complexity at most c and the homomorphism is given by sending each indeterminate Xi to an element of Z-complexity at most c.

Asymptotic properties. Let P be a property of noetherian local rings (possibly involving some additional data). We will use the phrase P holds asymptotically in mixed characteristic, to express that for each c, we can find a bound c0, such that if V is a complete discrete valuation ring of mixed characteristic and C a local V -affine algebra of V -complexity at most c (and a similar bound on the additional data), then property P holds for C, provided the characteristic of the residue field of V is at least c0. Sometimes, we have to control some additional invariants in terms of the bound c. In this paper, we will prove that in this sense, many Homological Conjectures hold asymptotically in mixed characteristic.

A final note. Its asymptotic nature is the main impediment of the present method to carry out Hochster’s program of obtaining tight closure and big Cohen–Macaulay algebras in mixed characteristic. For instance, despite the fact that we are able ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 431 to define an analogue of a balanced big Cohen–Macaulay algebra for O-affine domains, this object cannot be realized as an ultraproduct of ޚp-algebras, so that there is no candidate so far for a big Cohen–Macaulay in mixed characteristic. Although I will not pursue this line of thought in this paper, one could also define some nonstandard closure operation on ideals in O-affine algebras, but again, such an operation will only partially descend to any component.

Notation. A tuple x over a ring Z is always understood to be ﬁnite. Its length is denoted by |x| and the ideal it generates is denoted xZ. When we say that (Z, p) is local, we mean that p is its (unique) maximal ideal, but we do not imply that Z has to be noetherian. For a survey of the results and methods in this paper, see [Schoutens 2003b]. In the forthcoming [Schoutens 2004b] some of the present asymptotic versions will be generalized through a further investigation of the algebraic properties of ultraproducts using the notions introduced in Sections 5 and 6.

2. Ultraproducts

In this preliminary section, I state some generalities about ultraproducts and then briefly review the situation in equicharacteristic zero and the Ax–Kochen–Ershov Principle. The next section lays out the essential tools for conducting the transfer discussed in the introduction, to wit, approximations, protoproducts and nonstandard hulls, whose properties are then studied in Sections 5 and 6. The subsequent sections contain proofs of various asymptotic results, using these tools. Whenever we have an infinite index set W, we will equip it with some (unnamed) countably incomplete nonprincipal ultrafilter; ultraproducts will always be taken with respect to this ultrafilter and we will write

ulim Ow or simply O∞ w→∞ for the ultraproduct of objects Ow (this will apply to rings, ideals and elements alike). A first introduction to ultraproducts, including Łos’ Theorem, sufficient to understand the present paper, can be found in [Schoutens 2003d, §2]; for a more detailed treatment, see [Hodges 1993]. Łos’ Theorem states essentially that if a fixed algebraic relation holds among finitely many elements f1w,..., fsw in each ring Cw, then the same relation holds among their ultraproducts f1∞,..., fs∞ in the ultraproduct C∞, and conversely, if such a relation holds in C∞, then it holds in almost all Cw. Here almost all means “for all w in a subset of the index set which belongs to the ultrafilter” (the idea is that sets belonging to the ultrafilter are large, whereas the remaining sets are small). 432 HANS SCHOUTENS

An immediate, but important application of Łos’ Theorem is that the ultraproduct of algebraically closed fields of different prime characteristics is an (uncountable) algebraically closed field of characteristic zero, and any sufficiently large algebraically closed field of characteristic zero, including ރ, can be realized thus.1 This simple observation, in combination with work of van den Dries on nonstandard polynomials (see below), was exploited in [Schoutens 2003d] to define an alternative version of tight closure for ރ-affine algebras, called nonstandard tight closure, which was then further generalized to arbitrary noetherian local rings containing the rationals in [Aschenbrenner and Schoutens 2007]. The ensuing notions of F-regularity and F-rationality have proved to be more versatile [Schoutens 2004a; 2005a; 2005b] than those defined in [Hochster and Huneke 2000]. Let me briefly recall the results in [van den Dries and Schmidt 1984; van den Dries 1979] on nonstandard polynomials mentioned above. Let K w be fields (of arbitrary characteristic) with ultraproduct K ∞ (which is again a field by Łos’ The- orem). Let X be a fixed finite tuple of indeterminates and set A := K ∞[X] and Aw := K w[X]. Let A∞ be the ultraproduct of the Aw. As in the example discussed in the introduction, we have a canonical embedding of A inside A∞. In fact, the following easy observation, valid over arbitrary rings, describes completely the elements in A∞ that lie in A (the proof is straightforward and left to the reader).

Lemma 2.1. Let X be a ﬁnite tuple of indeterminates. Let Cw be rings and let C∞ be their ultraproduct. If f w is a polynomial in Cw[X] of degree at most c, for each w and for some c independent from w, then their ultraproduct in ulimw→∞ Cw[X] belongs already to the subring C∞[X], and conversely, every element in C∞[X] is obtained in this way. This result also motivates the notion of complexity from the introduction. Re- turning to the results of Schmidt and van den Dries, the following two properties of the embedding A ⊆ A∞ do not only imply the uniform bounds from [van den Dries and Schmidt 1984; Schoutens 2000a], but also play an important theoretical role in the development of nonstandard tight closure [Aschenbrenner and Schoutens 2007; Schoutens 2003d].

Theorem 2.2 (Schmidt and van den Dries). The embedding A ⊆ A∞ is faithfully ﬂat and every prime ideal in A extends to a prime ideal in A∞.

To carry out the present program, we have to replace the base ﬁelds K w by complete discrete valuation rings Ow. Unfortunately, we now have to face the following complications. Firstly, the ultraproduct O of the Ow is no longer noetherian, and

1To be more precise, any algebraically closed field of characteristic zero whose cardinality is of the form 2λ for some infinite cardinal λ, is an ultraproduct of algebraically closed fields of prime characteristic; under the generalized continuum hypothesis this means every uncountable algebraically closed field of characteristic zero. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 433 so in particular the corresponding A := O[X] is nonnoetherian. Moreover, the embedding A ⊆ A∞, where A∞ is now the ultraproduct of the Aw := Ow[X], although flat (see Theorem 4.2 below), is no longer faithfully flat (this is related to Kronecker’s problem; see [Aschenbrenner 2001a] or [Schoutens ≥ 2007] for details). Furthermore, not every prime ideal extends to a prime ideal. However, by working locally, we can circumvent all the latter complications (see Theorem 4.2 and Remark 4.5). To obtain the desired transfer, we will realize O in two different ways, as an ultraproduct of complete discrete valuation rings in positive characteristic and as an ultraproduct of complete discrete valuation rings in mixed characteristic, and then pass from one set to the other via O, as explained in the introduction (for more details, see Section 6.9 below). This is the celebrated Ax–Kochen–Ershov Principle [Ax and Kochen 1965; Ershov 1965; Ershov 1966], and I will discuss mix this now. For each p, let Op be a complete discrete valuation ring of mixed mix characteristic with residue field κ p of characteristic p. To each Op , we associate a corresponding equicharacteristic complete discrete valuation ring with the same residue field, by letting

eq (1) Op := κ p t J K where t is a single indeterminate. eq Theorem 2.3 (Ax–Kochen–Ershov). The ultraproduct of the Op is isomorphic (as mix a local ring) with the ultraproduct of the Op . Remark 2.4. As stated, we need to assume the continuum hypothesis. Otherwise, by the Keisler–Shelah Theorem [Hodges 1993, Theorem 9.5.7], one might need to take further ultrapowers, that is to say, over a larger index set. In order to not complicate the exposition, I will nonetheless make the set-theoretic assumption, so that our index set can always be taken to be the set of prime numbers. The reader can convince himself that all proofs in this paper can be adjusted so that they hold without any set-theoretic assumption. To conclude this section, I state a variant of Prime Avoidance which also works in mixed characteristic (note that for nonprime ideals one normally has to assume that the ring contains a field, see for instance [Eisenbud 1995, Lemma 3.3]). Proposition 2.5. Let Z be a local ring with infinite residue field κ. Let C be an arbitrary Z-algebra and let W be a finitely generated Z-submodule of C. If a1,..., at are ideals in C not containing W, then there exists f ∈ W not contained in any of the a j . Proof. We induct on the number t of ideals to be avoided, where the case t = 1 holds by assumption. Hence assume t > 1. By induction, we can find elements 434 HANS SCHOUTENS gi ∈ W, for i = 1, 2, which lie outside any a j for j 6= i. If either g1 ∈/ a1 or g2 ∈/ a2 we are done, so assume gi ∈ ai . Therefore, every element of the form g1 + zg2 with z a unit in Z does not lie in a1 nor in a2. Since κ is infinite, we can find t − 1 units z1, z2,..., zt−1 in Z whose residues in κ are all distinct. I claim that at least one of the g1 + zi g2 lies outside all a j , so that we found our desired element in W. Indeed, if not, then each g1 + zi g2 lies in one of the t − 2 ideals a3,..., at , by our previous remark. By the Pigeon Hole Principle, for some j and some l 6= k, we have that g1 + zk g2 and g1 + zl g2 lie both in a j . Hence so does their difference (zk − zl )g2. However, zk − zl is a unit in Z, by choice of the zi , so that g2 ∈ a j , contradiction. Corollary 2.6 (Controlled Ideal Avoidance). Let Z be a local ring with infinite residue field and let C be a (local) Z-affine algebra. If I and a1,..., at are ideals in C with I not contained in any ai , then I contains an element outside every ai . More precisely, if c is an upper bound for the Z-complexity of I , then there exists 2 an element f ∈ I of Z-complexity at most c , not contained in any ai .

Proof. Let (x1,..., xn) be a tuple of Z-complexity at most c generating I and let W be the Z-submodule of C generated by (x1,..., xn). In particular, W is not contained in any ai , so that we may apply Proposition 2.5 to obtain an element f ∈ W, outside each ai . Write f = z1x1 + ... zn xn with zi ∈ Z. After putting on a common denominator, we see that f has Z-complexity at most cn ≤ c2 (in case C is not local, the Z-complexity of f is in fact at most c). It is clear from the proof of Proposition 2.5 that in both results, we only need the residue ﬁeld to have a larger cardinality than the number of ideals to be avoided.

3. Approximations, protoproducts and nonstandard hulls

In this section, some general results on ultraproducts of finitely generated algebras over discrete valuation rings will be derived. We start with introducing some general terminology, over arbitrary noetherian local rings, but once we start proving some nontrivial properties in the next sections, we will specialize to the case that the base rings are discrete valuation rings. For some results in the general case, we refer to [Schoutens 2007; 2004b; ≥ 2007]. For each w, we fix a noetherian local ring Ow and let O be its ultraproduct. If the pw are the maximal ideals of the Ow, then their ultraproduct p is the maximal ideal of O. We will write IO for the ideal of infinitesimals of O, that is to say, the k intersection of all the powers p (note that in general IO 6= (0) and therefore, O is in particular nonnoetherian). By saturatedness of ultraproducts, O is quasicomplete in its p-adic topology in the sense that any Cauchy sequence has a (nonunique) limit. Hence the completion of O is O/IO (see also Lemma 5.3 below). Moreover, we will assume that all Ow ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 435 have embedding dimension at most . Hence so do O and O/IO. Since a complete local ring with finitely generated maximal ideal is noetherian [Matsumura 1986, Theorem 29.4], we showed that O/IO is a noetherian complete local ring. For more details in the case of interest to us, where each Ow is a discrete valuation ring or a field, see [Becker et al. 1979]. We furthermore fix throughout a tuple of indeterminates X = (X1,..., Xn), and we set A := O[X] and Aw := Ow[X]. Definition 3.1. The nonstandard O-hull of A is by definition the ultraproduct of the Aw and is denoted LO(A).

This terminology is a little misleading, because LO(A) does not only depend on O but also on the choice of Ow whose ultraproduct is O. In fact, we will exploit this dependence when applying the Ax–Kochen–Ershov principle, in which case we have to declare more precisely which nonstandard O-hull is meant. Nonetheless, whenever O and Ow are clear from the context, we will denote the nonstandard O-hull of A simply by L(A). By Łos’ Theorem, we have an inclusion O ⊆ L(A). Let us continue to write Xi for the ultraproduct in L(A) of the constant sequence Xi ∈ Aw. By Łos’ Theorem, the Xi are algebraically independent over O. In other words, A is a subring of L(A). In the next section, we will prove the key algebraic property of the extension A ⊆ L(A) when the base rings Ow are discrete valuation rings, to wit, its flatness. We start with extending the notions of nonstandard hull and approximation from [Schoutens 2003d], to arbitrary local O-affine algebras (recall that a local O-affine algebra is a localization of a finitely presented O-algebra at a prime ideal containing p).

O-approximations and nonstandard O-hulls. An O-approximation of a polynomial f ∈ A is a sequence of polynomials f w ∈ Aw, such that their ultraproduct is equal to f , viewed as an element in L(A). Note that according to Lemma 2.1, we can always find such an O-approximation. Moreover, any two O-approximations are equal for almost all w, by Łos’ Theorem. Similarly, an O-approximation of a finitely generated ideal I := fA with f a finite tuple, is a sequence of ideals I w := fw Aw, where fw is an O-approximation of f (meaning that each entry in fw is an O-approximation of the corresponding entry in f). Łos’ Theorem gives once more that any two O-approximations are almost all equal. Moreover, if I w is some O-approximation of I then

(2) ulim I w = I L(A). w→∞ Assume now that C is an O-affine algebra, say C = A/I with I a finitely generated ideal. We define an O-approximation of C to be the sequence of finitely 436 HANS SCHOUTENS generated Ow-algebras Cw := Aw/I w, where I w is some O-approximation of I . We define the nonstandard O-hull of C to be the ultraproduct of the Cw and denote it LO(C) or simply L(C). It is not hard to show that L(C) is uniquely defined up to C-algebra isomorphism (for more details see [Schoutens 2003d] or [Schoutens 2007]). From (2), it follows that L(C) = L(A)/I L(A). In particular, there is a canonical homomorphism C → L(C) obtained from the base change A → L(A). When I is not finitely generated, I L(A) might not be realizable as an ultraproduct of ideals, and consequently, has no O-approximation. Although one can find special cases of infinitely generated ideals admitting O-approximations, we will never have to do this in the present paper. Similarly, we only define O- approximations for O-affine algebras. Although A →L(A) is injective, this is not necessarily the case for C →L(C), if the Ow are not fields. For instance, if W is the set of prime numbers, Op := ޚp for p each p ∈ W and I = (1−π X, γ )A where π := ulimp→∞ p and γ := ulimp→∞ p , then I 6= (1) but I L(A) = (1). However, when the Ow are discrete valuation rings, we will see shortly, that this phenomenon disappears if we localize at prime ideals containing p. Next we define a process which is converse to taking O- approximations.

Protoproducts. Fix some c. For each w, let I w be an ideal in Aw of Ow-complexity at most c. In other words, we can write I w = fw Aw, for some tuple fw of Ow- complexity at most c. Let f be the ultraproduct of these tuples. By Lemma 2.1, the tuple f is already defined over A. We call I := fA the protoproduct of the I w. It follows that the I w are an O-approximation of I and that I L(A) is the ultraproduct of the I w. With Cw := Aw/I w and C := A/I , we call C the protoproduct of the Cw. The Cw are an O-approximation of C and their ultraproduct L(C) is the nonstandard O-hull of C. We can now extend the previous definition to the image in Cw of an element cw ∈ Aw (respectively, to the extension J wCw of a finitely generated ideal J w ⊆ Aw) of Ow-complexity at most c and define similarly their protoproduct c ∈ C and JC as the image in C of the respective protoproduct of the cw and the J w.

Functoriality. We have a commutative diagram ϕ C - D

(3) ? ? L(C) - L(D) L(ϕ) ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 437 where C → D is an O-algebra homomorphism of finite type between O-affine algebras and L(C) → L(D) is its base change over L(A). Alternatively, we may view this diagram coming from a sequence of Ow-algebras homomorphisms Cw → Dw of Ow-complexity at most c, for some c independent from w, in which case C → D and L(C) → L(D) are the respective protoproduct and ultraproduct of these homomorphisms. Lemma 3.2. Any prime ideal m of A containing p is finitely generated and its extension mL(A) is again prime. Proof. Since A/pA = κ[X] is noetherian, where κ is the residue field of O, the ideal m(A/pA) is finitely generated. Therefore so is m, since by assumption p is finitely generated. Moreover, L(A)/pL(A) is the ultraproduct of the κw[X], so that by Theorem 2.2, the extension m(L(A)/pL(A)) is prime, whence so is mL(A).

In particular, if mw is an O-approximation of m, then almost all mw are prime ideals. Therefore, the following notions are well-defined (with the convention that we put Bn equal to zero whenever n is not a prime ideal of the ring B). Let R be a local O-affine algebra, say, of the form Cm, with C an O-affine algebra and m a prime ideal containing p.

Deﬁnition 3.3. We call L(C)mL(C) the nonstandard O-hull of R and denote it LO(R) or simply L(R). Moreover, if Cw and mw are O-approximations of C and := m respectively, then the collection Rw (Cw)mw is an O-approximation of R.

One easily checks that the ultraproduct of the O-approximations Rw is precisely the nonstandard O-hull L(R).

4. Flatness of nonstandard O-hulls

In this section, we specialize the notions from the previous result to the situation where each Ow is a discrete valuation ring. We fix throughout the following notation. For each w, let Ow be a discrete valuation ring with uniformizing parameter π w and with residue field κw. Let O, π and κ be their respective ultraproducts, so that πO is the maximal ideal of O and κ its residue field. We call any ring of this form an ultra-DVR. The intersection of all π mO is called the ideal of infinitesimals m of O and is denoted IO. Using [Schoutens 1999], one sees that O/π O is an artinian local Gorenstein κ-algebra of length m. Fix a finite tuple of indeterminates X and let A := O[X]. As before, we denote the nonstandard O-hull of A by L(A); recall that it is given as the ultraproduct of the O-approximations Aw := Ow[X]. Proposition 4.1. For I an ideal in A, the residue ring A/I is noetherian if and only if IO ⊆ I . In particular, every maximal ideal of A contains IO and is of the form IO A + J with J a finitely generated ideal. 438 HANS SCHOUTENS

Proof. Let C := A/I for some ideal I of A. If C is noetherian, then the intersection n of all π C is zero by Krull’s Intersection Theorem. Hence IO ⊆ I . Conversely, if IO ⊆ I , then since A/IO A = (O/IO)[X] is noetherian, so is C. The last assertion is now clear. In spite of Lemma 3.2, there are even maximal ideals of A (necessarily not containing π) which do not extend to a proper ideal in L(A). For instance with X a single indeterminate and W = ގ, the ideal IO A + (1 − π X)A is maximal (with residue ﬁeld the ﬁeld of fractions of O/IO), but IOL(A) + (1 − π X)L(A) is the unit ideal. Indeed, let f ∞ be the ultraproduct of the

w f w := (1 − (π w X) )/(1 − π w X). w Since (1 − π w X) f w ≡ 1 modulo (π w) Aw, we get by Łos’ Theorem that (1 − π X) f ∞ ≡ 1 modulo IOL(A). Therefore, we cannot hope for A → L(A) to be faithfully flat. Nonetheless, using for instance a result of Aschenbrenner on bounds of syzygies, we do have this property for local affine algebras. This result will prove to be crucial in what follows. Theorem 4.2. The canonical homomorphism A → L(A) is flat. In particular, the canonical homomorphism of a local O-affine algebra to its nonstandard O-hull is faithfully flat, whence in particular injective. Proof. The last assertion is clear from the first, since the homomorphism R →L(R) is obtained as a base change of A → L(A) followed by a suitable localization, for any local O-affine algebra R. I will provide two different proofs for the first assertion For the first proof, we use a result of Aschenbrenner [Aschenbrenner 2001a] in order to verify the equational criterion for flatness, that is to say, given a linear equation L = 0, with L a linear form over A, and given a solution f∞ over L(A), we need to show that there exist solutions bi in A such that f∞ is an L(A)-linear combination of the bi . Choose Lw and fw with respective ultraproducts L and f∞. In particular, almost all Lw have Ow-complexity at most c, for some c independent from w. By Łos’ Theorem, fw is a solution of the linear equation Lw =0, for almost all w. Therefore, by [Aschenbrenner 2001a, Corollary 4.27], there is a bound 0 c , only depending on c, such that fw is an Aw-linear combination of solutions b1w,..., bsw of Ow-complexity at most c. Note that s can be chosen independent from w as well by [Schoutens 2007, Lemma 1]. In particular, the ultraproduct bi of the bi w lies in A by Lemma 2.1. By Łos’ Theorem, each bi is a solution of L = 0 in L(A), whence in A, and f∞ is an L(A)-linear combination of the bi , proving flatness. If we want to avoid the use of Aschenbrenner’s result, we can reason as follows. By Theorem 2.2, both extensions A/π A → L(A)/πL(A) and A⊗ Q → L(A)⊗ Q ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 439 are faithfully flat, where Q is the field of fractions of O. Let M be an A-module. Since π is A-regular, the standard spectral sequence

A/π A A A Torp (L(A)/πL(A), Torq (M, A/π A)) H⇒ Torp+q (L(A)/πL(A), M) degenerates into short exact sequences

A/π A : → A → Tori−1 (L(A)/πL(A), (0 M π)) Tori (L(A)/πL(A), M) A/π A Tori (L(A)/πL(A), M/π M), for all i ≥ 2. For i = 2, since A/π A → L(A)/πL(A) is ﬂat, the middle module A Tor2 (L(A)/πL(A), M) vanishes. Applying this to the short exact sequence π 0 → L(A) −→ L(A) → L(A)/πL(A) → 0 we get a short exact sequence

A A π A (4) 0 = Tor2 (L(A)/πL(A), M) → Tor1 (L(A), M)−→ Tor1 (L(A), M). On the other hand, flatness of A ⊗ Q → L(A) ⊗ Q yields A ⊗ = A⊗Q ⊗ ⊗ = (5) Tor1 (L(A), M) Q Tor1 (L(A) Q, M Q) 0. In order to prove that A →L(A) is flat, it suffices by [Matsumura 1986, Theorem A 7.8] to show that Tor1 (L(A), A/I ) vanishes, for every finitely generated ideal I A of A. Towards a contradiction, suppose that Tor1 (L(A), A/I ) contains a nonzero element τ. By (5), we have aτ = 0, for some nonzero a ∈ O. As observed in [Sabbagh 1974, Proposition 3], every polynomial ring over a valuation ring is coherent, so that in particular I is finitely presented (namely, since I is torsion-free over O, it is O-flat, and therefore finitely presented by [Raynaud and Gruson 1971, Theorem 3.4.6]). Hence we have some exact sequence

ϕ ϕ Aa2 −−→2 Aa1 −−→1 A → A/I → 0.

A Therefore Tor1 (L(A), A/I ) is calculated as the homology of the complex ϕ ϕ L(A)a2 −−→2 L(A)a1 −−→1 L(A).

a Suppose τ is the image of a tuple x ∈ L(A) 1 with ϕ1(x) = 0. Hence x does a not belong to ϕ2(L(A) 2 ) but ax does. Choose xw, aw and ϕi w with respective ultraproduct x, a and ϕi . By Łos’ Theorem, almost all xw lie in the kernel of ϕ1w but not in the image of ϕ2w, yet awxw lies in the image of ϕ2w. Choose nw ∈ ގ n maximal such that yw := (π w) w xw does not lie in the image of ϕ2w. Since almost all aw are nonzero, this maximum exists for almost all w. Therefore, if y is the a ultraproduct of the yw, then ϕ1(y) = 0 and y does not lie in ϕ2(L(A) 2 ), but πy 440 HANS SCHOUTENS

a2 A lies in ϕ2(L(A) ). Therefore, the image of y in Tor1 (L(A), A/I ) is a nonzero element annihilated by π, contradicting (4).

Remark 4.3. In [Schoutens ≥ 2007], I exhibit a general connection between the ﬂatness of an ultraproduct over certain canonical subrings and the existence of bounds on syzygies. In particular, using these ideas, the second argument in the above proof of ﬂatness reproves the result in [Aschenbrenner 2001a]. In fact, the role played here by coherence is not accidental either; see [Aschenbrenner 2001b] or [Schoutens ≥ 2007] for more details.

Theorem 4.4. Let R be a local O-afﬁne algebra with nonstandard O-hull L(R) and O-approximation Rw.

• Almost all Rw are ﬂat over Ow if and only if R is torsion-free over O if and only if π is R-regular.

• Almost all Rw are domains if and only if R is.

Proof. Suppose first that almost all Rw are flat over Ow, which amounts in this case, to almost all Rw being torsion-free over Ow. By Łos’ Theorem, L(R) is torsion- free over O, and since R ⊆ L(R), so is R. Conversely, assume π is R-regular. By faithful flatness, π is L(R)-regular, whence almost all π w are Rw-regular by Łos’ Theorem. Since the Ow are discrete valuation rings, this means that almost all Ow → Rw are flat. If almost all Rw are domains, then so is L(R) by Łos’ Theorem, and hence so is R, since it embeds in L(R). Conversely, assume R is a domain. If π = 0 in R, then L(R) is a domain by Lemma 3.2, whence so are almost all Rw by Łos’ Theorem. So assume π is nonzero in R, whence R-regular. By what we just proved, R is then torsion-free over O. Let Q be the field of fractions of O. Write R in the form S/p, where S is some localization of A at a prime ideal containing π and p is a finitely generated prime ideal in S. Since S/p is torsion-free over O, the extension p(S ⊗O Q) is again prime and its contraction in S is p. By Theorem 2.2, since we are now over a field, p(L(S)⊗O Q) is a prime ideal, where L(S) is the nonstandard O-hull of S (note that L(S) ⊗O Q is then the nonstandard hull of S ⊗O Q in the sense of [Schoutens 2003d]). Moreover, since S/p is torsion-free over O, so is L(S)/pL(S) by the first assertion. This in turn means that

pL(S) = p(L(S) ⊗O Q) ∩ L(S), showing that pL(S) is prime. It follows then from Łos’ Theorem that almost all pw are prime, where pw is an O-approximation of p, and hence almost all Rw are domains. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 441

Remark 4.5. The last assertion is equivalent with saying that any prime ideal in R extends to a prime ideal in L(R). Indeed, let q be a prime ideal in R with O-approximation qw. By the preceding result (applied to R/q and its O- approximation Rw/qw), we see that almost all qw are prime, whence so is their ultraproduct qL(R), by Łos’ Theorem.

5. Geometric dimension

In this and the next section, we will study the local algebra of the category Aff(O). Although part of the theory can be developed for arbitrary base rings O, or even for arbitrary local rings of finite embedding dimension (see [Schoutens 2004b]), we will only deal with the case that O is a local domain of embedding dimension one. Recall that the embedding dimension of a local ring (Z, p) is by definition the minimal number of generators of p, and its ideal of infinitesimals IZ is the intersection of all powers pn. Of course, if Z is moreover noetherian, then its ideal ˜ of infinitesimals is zero. In general, we call Z := Z/IZ the separated quotient of Z. For the duration of the next two sections, let O denote a local domain of embedding dimension one, with generator of the maximal ideal π, with ideal of infinitesimals IO and with residue field κ. We will work in the category Aff(O) of local O-affine algebras, that is to say, the category of algebras of the form R := (A/I )m, where as before A := O[X] for some finite tuple of indeterminates X, where I is a finitely generated ideal in A and where m is a prime ideal containing π and I . Nonetheless, some results can be stated even for local algebras which are locally finitely generated over O, that is, without the assumption that I is finitely generated. We call R a torsion-free O-algebra if it is torsion-free over O (that is to say, if ar = 0 for some r ∈ R and some nonzero a ∈ O, then r = 0). Recall from Theorem 4.4 that a local O-affine algebra R is torsion-free if and only if π is R-regular.

Lemma 5.1. The separated quotient O/IO of O is a discrete valuation ring with uniformizing parameter π.

Proof. For each element a ∈ O outside IO, there is a smallest e ∈ ގ for which a ∈/ π e+1O. Hence a = uπ e with u a unit in O. It is now straightforward to check that the assignment a 7→ e induces a discrete valuation on O/IO. Note that we do not even need O to be domain, having positive depth (that is to say, assuming that πO is not an associated prime of O; see [Bruns and Herzog 1993, Proposition 9.1.4]) would sufﬁce, for then π is necessarily O-regular. How- ever, we do not need this amount of generality as in all our applications O will be an ultra-DVR, that is to say, an ultraproduct of discrete valuation rings Ow. If we are in this situation, then as before, we let Aw := Ow[X] and we let L(A) be their ultraproduct. Moreover, for R = (A/I )m as above, we let L(R) := (L(A)/I L(A))mL(A) 442 HANS SCHOUTENS

:= be its nonstandard O-hull and we let Rw (Aw/I w)mw be an O-approximation of R, where I w and mw are O-approximations of I and m respectively. Note that m is finitely generated, as it contains by definition π. Lemma 5.2. Let (R, m) be a local ring which is locally finitely generated over O. If I is a proper ideal in R containing some power π m, then the intersection of all n I for n ∈ ގ is equal to IO R. In particular, IR = IO R and the separated quotient ˜ of R is equal to R := R/IO R whence is noetherian. Proof. Suppose π m ∈ I ⊆ m. Let J be the intersection of all I n for n ∈ ގ. m ˜ Since π ∈ I , we get IO R ⊆ J. Since R is locally finitely generated over the discrete valuation ring O/IO (see Lemma 5.1), it is noetherian. Applying Krull’s Intersection Theorem (see for instance [Matsumura 1986, Theorem 8.10]), we get ˜ J R = (0), and hence that J = IO R. The last assertion follows by letting I := m. Lemma 5.3. Let O be an ultra-DVR. A local O-affine algebra (R, m) has the same m-adic completion as its separated quotient, and this is also isomorphic to L(R)/IL(R). In particular, the completion is noetherian. ˜ Proof. Let R := R/IR be the separated quotient. For every n, we have R/mn =∼ R˜/mn R˜ =∼ L(R)/mnL(R), where the second isomorphism follows from the fact that length is a first order invariant (see for instance [Schoutens 1999]). Hence R, R˜ and L(R) have the same completion Rb. Noetherianness now follows from Lemma 5.2. By saturatedness of ultraproducts (with respect to a countably incomplete nonprincipal ultrafilter), L(R) is quasicomplete in the sense that every Cauchy sequence has a (nonunique) limit. Therefore, its separated quotient L(R)/IL(R) is complete, whence equal to Rb. For a more detailed proof, see [Schoutens 2004b, Lemma 5.2]. Our first goal is to introduce a good notion of dimension. Below, the dimension of a ring will always mean its Krull dimension. Recall that it is always finite for noetherian local rings. Theorem 5.4. For a local ring (R, m) which is locally finitely generated over O, the following numbers are all equal: • the least possible length d of a tuple in R generating some m-primary ideal; • the dimension db of the completion Rb; ˜ ˜ • the dimension d of the separated quotient R := R/IO R;

• the degree d of the Hilbert–Samuel polynomial χR, which is deﬁned as the unique polynomial with rational coefﬁcients for which χR(n) equals the length of R/mn+1 for all large n. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 443

If π is R-regular, then R/π R has dimension d − 1. If , moreover, O is an ultra-DVR and R a torsion-free local O-afﬁne algebra with O-approximation Rw, then almost all Rw have dimension d.

Proof. By Lemmas 5.2 and 5.3, the separated quotient R˜ is noetherian, with com- = ˜ = pletion equal to Rb. Hence db d. Moreover, χR χRb, so that by the Hilbert–Samuel theory, d = db. Let x be a tuple of length d˜ such that its image in R˜ is a system of parameters ˜ n of R. Hence, for some n, we have that m ⊆ xR + IO R. In particular, since n+1 n n+1 IO R ⊆ π R, we can find x ∈ xR and r ∈ R, such that π = x +rπ . Therefore, n n n π ∈ xR, since 1 − rπ is a unit. Since IO ⊆ π O, we get m ⊆ xR, showing that xR is an m-primary ideal and hence that d ≤ d˜. On the other hand, if y is a tuple of length d such that yR is m-primary, then yR˜ is an mR˜-primary ideal, and hence d˜ ≤ d. This concludes the proof of the first assertion. Assume that π is moreover R-regular. I claim that π is R˜-regular. Indeed, ˜ n suppose πr˜ = 0, for some r˜ ∈ R. Take a preimage r ∈ R, so that πr ∈ IO R ⊆ π R, n−1 for every n. Since π is R-regular, we get r ∈ π R, for all n. Therefore, r ∈ IO R by Lemma 5.2, whence r˜ = 0 in R˜, as we needed to show. Since π is R˜-regular and R˜/π R˜ = R/π R, the dimension of R/π R is d˜ − 1. Suppose finally that O is moreover an ultra-DVR. We already observed that Rw/π w Rw is an approximation of R/π R in the sense of [Schoutens 2003d]. In particular, by [Schoutens 2003d, Theorem 4.5], almost all Rw/π w Rw have dimen- ˜ sion d −1. Since π is L(R)-regular by flatness, whence π w is Rw-regular by Łos’ ˜ Theorem, we get that Rw has dimension d, for almost all w.

5.5. Geometric dimension. The common value given by the theorem is called the geometric dimension of R. We call a tuple x in R generic, if it generates an m- primary ideal and has length equal to the geometric dimension of R. Note that if (x1,..., xd ) is a generic sequence, then R/(x1,..., xe)R has geometric dimension d − e.

Corollary 5.6. In a local ring (R, m) which is locally ﬁnitely generated over O, every m-primary ideal contains a generic sequence. ˜ Proof. Let R := R/IO R and let d be the geometric dimension of R. Let n be an m-primary ideal of R. Since nR˜ is mR˜-primary and R˜ is noetherian, we can ﬁnd a tuple y with entries in n so that its image in R˜ is a system of parameters. In ˜ particular, y has length d by Theorem 5.4. Let S := R/yR and S := S/IO S. By Theorem 5.4, the geometric dimension of S is equal to the dimension of S˜, whence is zero since S˜ = R˜/yR˜. In particular, yR is m-primary. Since y has length equal to the geometric dimension of R, it is therefore a generic sequence. 444 HANS SCHOUTENS

In fact the above proof shows that there is a one-one correspondence between generic sequences in R and systems of parameters in R/IO R. In general, the last assertion in Theorem 5.4 is false when R is not torsion-free. For instance, let R := O/aO with a a nonzero infinitesimal, so that each Rw = Ow/awOw has dimension zero, but R/IR is the (one-dimensional) discrete valuation ring O/IO. In the following definition, let O be an ultra-DVR and let R be a local O-affine algebra of geometric dimension d, with O-approximation Rw. Note that the Rw have almost all dimension at most d. Indeed, if y has length d and generates an m-primary ideal, then almost all yw are mw-primary by Łos’ Theorem, for yw an O-approximation of y.

Definition 5.7. We say that R is isodimensional if almost all Rw have dimension equal to the geometric dimension of R. Theorem 5.4 shows that every torsion-free local O-affine algebra is isodimensional. In particular, over an ultra-DVR, the protoproduct R of domains Rw of uniformly bounded Ow-complexity is isodimensional, since L(R) is then a domain by Łos’ Theorem, whence so is R as it embeds in L(R). The next result shows that generic sequences in an isodimensional ring are the analog of systems of parameters. Corollary 5.8. Let O be an ultra-DVR and R an isodimensional local O-affine algebra with O-approximation Rw. Let x be a tuple in R with O-approximation xw. If x is generic, then xw is a system of parameters of Rw, for almost all w. Con- c versely, if (π w) ∈ xw Rw, for some c and almost all w, then x is generic.

Proof. Let m be the maximal ideal of R, with O-approximation mw. Let d be the geometric dimension of R, so that almost all Rw have dimension d. Suppose first that x is generic, so that |x| = d and xR is m-primary. Since xL(R) is then mL(R)- primary, xw Rw is mw-primary by Łos’ Theorem, showing that xw is a system of parameters for almost all w. Conversely, suppose xw is a system of parameters of Rw, generating an ideal c c containing (π w) . By Łos’ Theorem and faithful flatness, π ∈ xR. Applying c [Schoutens 2007, Corollary 4] to the artinian base ring Ow/(π w) , we can find a 0 c0 bound c , only depending on c, such that (mw) ⊆ xw Rw, for almost all w. Hence 0 mc L(R) ⊆ xL(R), so that by faithful flatness, xR is m-primary. This shows that x is generic. The additional requirement in the converse is necessary: indeed, for arbitrary n nw > 0, the element (π w) w is a parameter in Ow and has Ow-complexity zero, but if nw is unbounded, its ultraproduct is an infinitesimal whence not generic. To characterize isodimensional rings, we use the following notion introduced in [Schoutens 2006]. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 445

Definition 5.9 (Parameter degree). The parameter degree of a noetherian local ring C is by definition the smallest possible length of a residue ring C/xC, where x runs over all systems of parameters of C. In general, the parameter degree is larger than the multiplicity, with equality precisely when C is Cohen–Macaulay, provided the residue field is infinite (see [Matsumura 1986, Theorem 17.11]). The homological degree of C is an upper bound for its parameter degree (see [Schoutens 2006, Corollary 4.6]). A priori, being isodimensional is a property of the O-approximations of R, of for that matter, of its nonstandard O-hull. However, the last equivalent condition in the next result shows that it is in fact an intrinsic property. Proposition 5.10. Let O be an ultra-DVR and let R be a local O-affine algebra with O-approximation Rw. The following are equivalent: (i) R is isodimensional; (ii) there exists a c ∈ ގ, such that for almost all w, we can find a system of parameters xw of Rw of Ow-complexity at most c, generating an ideal containing c (π w) ;

(iii) there exists an e ∈ ގ, such that almost all Rw have parameter degree at most e; (iv) for every generic sequence in R of the form (π, y), the contracted ideal yR∩O is zero.

Proof. Let m be the maximal ideal of R, with O-approximation mw. Let d be the 0 geometric dimension of R and let d be the dimension of almost all Rw. Suppose 0 ﬁrst that d = d . Let x be any generic sequence in R with O-approximation xw. By Łos’ Theorem, almost all xw generate an mw-primary ideal. Since their length is equal to the dimension of Rw, they are almost all systems of parameters of Rw. Choose c large enough so that π c ∈ xR. Enlarging c if necessary, we may moreover assume by Lemma 2.1 that almost all xw have Ow-complexity at most c. By Łos’ c Theorem, (π w) ∈ xw Rw, so (ii) holds. c Assume next that c and the xw are as in (ii). Let Rw := Rw/(π w) Rw. We can c apply [Schoutens 2007, Corollary 2] over Ow/(π w) Ow to the mw Rw-primary 0 ideal xw Rw, to conclude that there is some c , depending only on c, such that 0 Rw/xw Rw has length at most c . Since the latter residue ring is just Rw/xw Rw by 0 assumption, the parameter degree of Rw is at most c , and hence (iii) holds. To show that (iii) implies (i), assume that almost all Rw have parameter degree at most e. Let yw be a system of parameters of Rw such that Rw/yw Rw has length e at most e, for almost all w. It follows that (mw) is contained in yw Rw. Let y∞ e be the ultraproduct of the yw. By Łos’ Theorem, m L(R) ⊆ y∞L(R) whence e m Rb ⊆ y∞ Rb, by Lemma 5.3, showing that y∞ Rb is mRb-primary. Since y∞ has 446 HANS SCHOUTENS

0 length at most d (some entries might be zero in Rb), the dimension of Rb is at most d0. Since we already remarked that d0 ≤ d, we get from Theorem 4.4 that d0 = d. So remains to show that (iv) is equivalent to the other conditions. Assume first that it holds but that R is not isodimensional. Since we have inequalities d − 1 ≤ d0 ≤d, this means that d0 =d−1. Moreover, R/π R must have geometric dimension also equal to d −1, for if not, its geometric dimension would be d, whence almost all Rw/π w Rw would have dimension d by [Schoutens 2003d, Theorem 4.5], which is impossible. Since there is a uniform bound c on the Ow-complexity of each Rw, we can choose, using Corollary 2.6, a system of parameters yw of Rw of Ow- 2 complexity at most c . In particular, some power of π w lies in yw Rw. Let a ∈ O be the ultraproduct of these powers. If y is the ultraproduct of the yw, then y is already defined over R by Lemma 2.1. By Łos’ Theorem, a ∈ yL(R), whence by faithful flatness, a is a nonzero element in yR ∩ O. Therefore, to reach the desired contradiction with (iv), we only need to show that (π, y) is generic. As we already established, Rw/π w Rw has dimension d − 1, so that yw is also a system of parameters in that ring. Therefore, y is a system of parameters in R/π R by [Schoutens 2003d, Theorem 4.5]. This in turn implies that (π, y) generates an m- primary ideal in R. Since this tuple has length d, it is therefore generic, as we wanted to show. Finally, assume R is isodimensional, and suppose (π, y) is generic. Let a ∈ yR ∩O and choose O-approximations aw and yw of a and y respectively. By Łos’ Theorem, aw ∈ yw Rw. However, if a is nonzero, then aw is, up to a unit, a power of π w, which contradicts the assertion in Corollary 5.8 that (π w, yw) is a system of parameters. So a = 0, as we needed to show.

Corollary 5.11. For each c, there exists a bound PD(c) with the following property. Let V be a discrete valuation ring and let C be a local V -afﬁne algebra of V - complexity at most c. If C is torsion-free over V , then the parameter degree of C is at most PD(c).

Proof. If the statement is false for some c, then we can ﬁnd for each w a discrete valuation ring Ow and a torsion-free local Ow-afﬁne algebra Rw of Ow-complexity at most c, whose parameter degree is at least w. Let R be the protoproduct of the Rw and let L(R) be their ultraproduct. Since π w is Rw-regular, π is L(R)- regular, whence R-regular. Hence R is isodimensional by Theorem 5.4. Therefore, there is a bound on the parameter degree of almost all Rw by Proposition 5.10, contradicting our assumption.

Our next goal is to introduce a notion similar to height. Let I be an arbitrary ideal of R. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 447

Deﬁnition 5.12 (Geometric height). We call the geometric height of I the maximum of all h such that there exists a generic sequence whose ﬁrst h entries belong to I .

For noetherian rings, we cannot expect a good relationship between the height of an ideal and the dimension of its residue ring, unless the ring is a catenary domain; the following is the analogue over ultra-DVR’s.

Theorem 5.13. Let O be an ultra-DVR and let R be a local O-afﬁne domain with O-approximation Rw. Let I be a ﬁnitely generated ideal in R with O-approximation I w. If R/I is isodimensional, then the geometric height of I is equal to the geometric dimension of R minus the geometric dimension of R/I , and this is also equal to the height of almost all I w.

Proof. Let d be the geometric dimension of R and e the geometric dimension of R/I . Since a domain is isodimensional, almost all Rw have dimension d by Theorem 5.4, and by assumption, almost all Rw/I w have dimension e. Let h be the geometric height of I . Let z be a generic sequence in R with its first h entries in I , and let zw be an O-approximation of z. By Corollary 5.8, almost all zw are a system of parameters in Rw. Since by Łos’ Theorem the first h entries of zw lie in I w, we get that Rw/I w has dimension at most d − h. In other words, h ≤ d − e. Since almost all Rw are catenary domains, almost all I w have height d − e. So remains to show that d−e ≤ h. By Lemma 5.2, the separated quotient of R/I is equal to R˜/I R˜. Therefore, by the remark following Corollary 5.6, we can find a generic sequence (x1,..., xd ) in R such that (the image of) (x1,..., xe) is a generic sequence in R/I . By definition of generic sequence, S := R/(x1,..., xe)R has geometric dimension d − e. If xi w is an O-approximation of xi , then almost each xw := (x1w,..., xew) is a system of parameters in Rw/I w by Corollary 5.8. Since xw is therefore part of a system of parameters in Rw, almost each Sw := Rw/xw Rw has dimension d − e by [Matsumura 1986, Theorem 14.1]. By choice of the xi , the ideal I + (x1,..., xe)R is m-primary and hence IS is mS-primary. Therefore, by Corollary 5.6, we can find a tuple y of length d − e in I , so that its image in S is a generic sequence. It follows that (x1,..., xe)R + yR is m-primary. Since (y, x1,..., xe) has length d, it is a generic sequence, showing that d − e ≤ h.

6. Pseudo singularities

In this section, we maintain the notation introduced in the previous section. Our goal is to extend several singularity notions of noetherian local rings to the category of local O-afﬁne algebras. 448 HANS SCHOUTENS

Grade and depth. Let B be an arbitrary ring and I := (x1,..., xn)B a finitely generated ideal. The grade of I , denoted grade(I ), is by definition equal to n − h, where h is the largest value i for which the i-th Koszul homology Hi (x1,..., xn) is nonzero. For a local ring R of finite embedding dimension, we define its depth as the grade of its maximal ideal. If B is moreover noetherian, then we can define the grade of I alternatively as the i minimal i for which ExtB(B/I, B) is nonzero (for all this see for instance [Bruns and Herzog 1993, §9.1]). An arbitrary local ring has positive depth if and only if its maximal ideal is not an associated prime. Grade, and hence depth, deforms well, in the sense that the

grade(I (B/xB)) = grade(I ) − |x| for every B-regular sequence x in I . For a locally finitely generated O-algebra (R, m), its depth never exceeds its geometric dimension. Indeed, by definition, the grade of a finitely generated ideal never exceeds its minimal number of generators, and by [Bruns and Herzog 1993, Proposition 9.1.3], the depth of R is equal to the grade of any of its m-primary ideals. It follows that the depth of R is at most its geometric dimension. In general, the grade of a finitely generated ideal might be positive without it containing a B-regular element. However, the next lemma shows that this is not the case for ultraproducts of noetherian local rings.

Lemma 6.1. Let C∞ be the ultraproduct of noetherian local rings Cw and let I ∞ be a ﬁnitely generated ideal of C∞ obtained as the ultraproduct of ideals I w ⊆ Cw. If I ∞ has grade n, then there exists a C∞-regular sequence of length n with all of its entries in I ∞. Moreover, any permutation of a C∞-regular sequence is again C∞-regular.

Proof. By [Bruns and Herzog 1993, Proposition 9.1.3], there exists a finite tuple of indeterminates Y and a C∞[Y ]-regular sequence f∞ of length n, with all of its entries in I ∞C∞[Y ]. Choose tuples fw in Cw[Y ] so that their ultraproduct is f∞. By Łos’ Theorem, fw is Cw[Y ]-regular and has all of its entries in I wCw[Y ], for almost all w. This shows that I wCw[Y ] has grade at least n. Since Cw → Cw[Y ] is faithfully flat, I w has grade at least n by [Bruns and Herzog 1993, Proposition 9.1.2]. Hence, since Cw is noetherian, we can find a Cw-regular sequence xw of length n with all of its entries in I w. By Łos’ Theorem, the ultraproduct x∞ of the xw is C∞-regular and has all of its entries in I ∞. The last assertion follows from Łos’ Theorem and the fact that in a noetherian local ring, any permutation of a regular sequence is again regular ([Matsumura 1986, Theorem 16.3]). ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 449

Recall that a noetherian local ring for which its dimension and its depth (respectively, its dimension and its embedding dimension) coincide is Cohen–Macaulay (respectively, regular). We will shortly see that upon replacing dimension by geometric dimension, we get equally well behaved notions. Let us therefore make the following definitions, for R a local O-affine algebra. Definition 6.2. We say that R is pseudo-Cohen–Macaulay, if its geometric dimension is equal to its depth, and pseudoregular, if its geometric dimension is equal to its embedding dimension. Theorem 6.3. Let O be an ultra-DVR and let R be an isodimensional local O- affine algebra with O-approximation Rw. In order for R to be pseudo-Cohen– Macaulay it is necessary and sufficient that almost all Rw are Cohen–Macaulay. Proof. Let d be the geometric dimension of R and δ its depth. Suppose first that d = δ. Since R → L(R) is faithfully flat, L(R) has depth δ as well by [Bruns and Herzog 1993, Proposition 9.1.2]. By Lemma 6.1, there exists an L(R)-regular sequence x∞ of length d. If xw is an O-approximation of x∞, then almost each xw is Rw-regular by Łos’ Theorem. Since almost all Rw have dimension d by isodimensionality, almost all are Cohen–Macaulay. Conversely, assume almost all Rw are Cohen–Macaulay. It follows by reversing the above argument that L(R) has depth d and hence, so has R, by faithful flatness. Since every system of parameters is a regular sequence in a local Cohen–Mac- aulay ring, we expect a similar behavior for generic sequences, and this indeed holds. Theorem 6.4. Let O be an ultra-DVR and let R be an isodimensional local O- affine algebra. If R is pseudo-Cohen–Macaulay, then any generic sequence is R-regular.

Proof. Let x be a generic sequence with O-approximation xw. Almost each xw is a system of parameters in Rw, by Corollary 5.8. Since almost all Rw are Cohen– Macaulay by Theorem 6.3, almost each xw is Rw-regular. Hence x is L(R)-regular, by Łos’ Theorem, whence R-regular, by faithful ﬂatness. Theorem 6.5. Let O be an ultra-DVR. An isodimensional local O-afﬁne algebra R with O-approximation Rw is pseudoregular if and only if almost all Rw are regular local rings.

Proof. Let m be the maximal ideal of R, with O-approximation mw. Let L(R) be the nonstandard O-hull of R. Let be the embedding dimension of R and d its geometric dimension. Suppose that R is pseudoregular, that is to say, that = d. Hence m = xR for some d-tuple x, necessarily generic. Since mL(R) = xL(R), 450 HANS SCHOUTENS

Łos’ Theorem yields that mw = xw R, where xw is an O-approximation of x. Since almost all Rw have dimension d, almost all are regular local rings. Conversely, suppose almost all Rw are regular. Since the Ow-complexity of almost all Rw is at most c, for some c, we can find a regular system of parameters xw of Ow-complexity at most c (as part of a minimal system of generators of mw). By Lemma 2.1, their ultraproduct x belongs to R, and is a generic sequence by Corollary 5.8. By Łos’ Theorem and faithful flatness, xR = m whence ≤ d. Since geometric dimension never exceeds embedding dimension, = d and R is pseudoregular. The following is now immediate from the previous result and Theorem 4.4. Corollary 6.6. Let O be an ultra-DVR. If R is a pseudoregular local O-affine algebra, then R is a domain if and only if it is isodimensional. Moreover, if this is the case, then every localization of R with respect to a prime ideal containing π is again pseudoregular.

In fact, the protoproduct R of regular local Ow-afﬁne algebras Rw of uniformly bounded Ow-complexity is pseudoregular and isodimensional. Indeed, we already observed that then R is isodimensional, and therefore by Theorem 6.5, pseudoregular. For a homological characterization of pseudoregularity, see Corollary 11.5 below. Example 6.7. If R denotes the localization of O[X, Y ]/(X 2 + Y 3 + π) at the maximal ideal generated by X, Y and π, then R is pseudoregular (namely X and Y generate the maximal ideal, so = 2, and since R/π R has dimension one, d = 2 as well). Note though that R/π R is not regular. Corollary 6.8. Let O be an ultra-DVR and let R be an isodimensional local O- afﬁne algebra. If R is pseudoregular, then it is pseudo-Cohen–Macaulay.

Proof. Let Rw be an O-approximation of R. By Theorem 6.5, almost all Rw are regular whence Cohen–Macaulay. This in turn implies that R is pseudo-Cohen– Macaulay by Theorem 6.3. Without the isodimensionality assumption, the result is false. For instance, let a be a nonzero element in the ideal of inﬁnitesimals of O and put R := O/aO. It follows that R has geometric dimension one, whence is pseudoregular, but its depth is zero.

6.9. Transfer. Let me now elaborate on why the results in this section are instances of transfer between positive and mixed characteristic. Suppose O˜ is a ˜ second ultra-DVR, realized as the ultraproduct of discrete valuation rings Ow and ∼ ˜ ˜ suppose O = O. Note that this does not imply that Ow and Ow are almost all pair-wise isomorphic. In fact, in the next sections, one set of discrete valuation ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 451 rings will be of mixed characteristic and the other set of prime characteristic. Let R be a local O-affine algebra. Since R is then also local O˜ -affine, its admits a nonstandard O˜ -hull and O˜ -approximations with respect to this second set of dis- ˜ crete valuation rings; let us denote them by LO˜ (R) and Rw respectively. Suppose ˜ Ow and Ow have pair-wise isomorphic residue fields (as will be the case below). Since the Rw/π w Rw are an approximation of the κ-algebra R/π R (in the sense of ˜ ˜ [Schoutens 2003d]) and, mutatis mutandis, so are the Rw/π˜ w Rw, where π˜ w is a ˜ uniformizing parameter of Ow, we get from [Schoutens 2003d, 3.2.3] that almost ˜ ˜ all Rw/π w Rw are isomorphic to Rw/π˜ w Rw. Therefore, if we assume that there ˜ is no torsion, then Rw and Rw have the same dimension, and one set consists of almost all Cohen–Macaulay local rings if and only if the other set does (note that this argument does not yet use the above pseudo notions). However, this argument breaks down in the presence of torsion, or, when we want to transfer the regularity property. This can be overcome by using the notions defined in this section, provided we have a uniform upper bound on the parameter degree. Suppose, for some d, e ∈ ގ, that almost all Rw have dimension d and parameter degree at most e. Note that in view of Corollary 5.11 this last condition is automati- cally satisfied if almost all Rw are torsion-free over Ow; and that it is implied by the assumption that almost all Rw have uniformly bounded homological multiplicity (see [Schoutens 2006, Corollary 4.6]). Applying Proposition 5.10 twice gives first ˜ that R is isodimensional, with geometric dimension d, and then that almost all Rw have dimension d and uniformly bounded parameter degree. Now, Theorems 6.3 and 6.5 tell us that almost all Rw are respectively Cohen–Macaulay or regular, if ˜ and only if almost all Rw are.

7. Big Cohen–Macaulay algebras

In [Aschenbrenner and Schoutens 2007; Schoutens 2004a], ultraproducts of absolute integral closures in characteristic p were used to define big Cohen–Macaulay algebras in equicharacteristic zero. This same process can be used in the current mixed characteristic setting. Recall that for an arbitrary domain B, we define its absolute integral closure as the integral closure of B in some algebraic closure of its field of fractions and denote it B+. This is uniquely defined up to B-algebra isomorphism. mix For each prime number p, let Op be a mixed characteristic complete discrete valuation ring with uniformizing parameter π p and residue field κ p of character- eq istic p, and let O, π and κ be their respective ultraproducts. Put Op := κ p t , for t a single indeterminate. By Theorem 2.3, the Ax–Kochen–Ershov Theorem,J K eq O is isomorphic to the ultraproduct of the Op . As before, IO denotes the ideal of infinitesimals of O. Put A := O[X], for a fixed tuple of indeterminates X, 452 HANS SCHOUTENS

eq mix and let LO (A) and LO (A) be its respective equicharacteristic and mixed characteristic nonstandard O-hull, that is to say, the ultraproduct of respectively the eq eq mix mix A p := Op [X] and the A p := Op [X]. eq eq Throughout, R will be a local O-affine domain with R p and LO (R) respectively an equicharacteristic O-approximation and the equicharacteristic nonstandard O- eq eq hull of R (so that LO (R) is the ultraproduct of the R p ). By Theorem 4.4, almost eq all R p are local domains. eq + Definition 7.1. Define Ꮾ(R) as the ultraproduct of the (R p ) . eq + eq Since (R p ) is well-defined up to R p -algebra isomorphism, we have that Ꮾ(R) is well-defined up to R-algebra isomorphism. Moreover, this construction is weakly functorial in the following sense. Let R → S be an O-algebra homomorphism be- eq eq tween local O-affine domains. This induces Op -algebra homomorphisms R p → eq S p of the corresponding equicharacteristic O-approximations. These in turn yield eq + eq + homomorphisms (R p ) → (S p ) between the absolute integral closures. Taking ultraproducts, we get an O-algebra homomorphism Ꮾ(R) → Ꮾ(S) and a commutative diagram

R - S

(6) ? ? Ꮾ(R) - Ꮾ(S).

Theorem 7.2. If R is a local O-afﬁne domain, then any generic sequence in R is Ꮾ(R)-regular. eq eq Proof. Let LO (R) and R p be respectively, the equicharacteristic nonstandard O- hull and an equicharacteristic O-approximation of R. Let x be a generic sequence, and let xp be an O-approximation of x. By Corollary 5.8, almost each xp is a eq eq + system of parameters in R p , whence is (R p ) -regular by [Hochster and Huneke 1992]. By Łos’ Theorem, x is Ꮾ(R)-regular.

8. Improved New Intersection Theorem

The remaining sections will establish various asymptotic versions in mixed characteristic of the Homological Conjectures listed in the abstract. We start with discussing Intersection Theorems. By [Roberts 1987], we now know that the New Intersection Theorem holds for all noetherian local rings. However, this is not yet known for the Improved New Intersection Theorem. We need some terminology and notation (all taken from [Bruns and Herzog 1993]). ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 453

Let C be an arbitrary noetherian local ring and ϕ : Ca → Cb a linear map between ﬁnite free C-modules. We will always think of ϕ as an (a × b)-matrix over C. For r > 0, recall that the r-th Fitting ideal of ϕ, denoted Ir (ϕ), is the ideal in C generated by all (r × r) minors of ϕ; if r exceeds the size of the matrix, we put Ir (ϕ) := (0). By a ﬁnite free complex over C we mean a complex

a ϕs a − ϕs−1 ϕ2 a ϕ1 a (F•) 0 → C s −−→C s 1 −−−→· · · −−→C 1 −−→C 0 → 0. We call s the length of the complex, and for each i, we deﬁne s X j−i ri := (−1) a j . j=i

We will refer to ri as the expected rank of ϕi . We will call the residue ring C/Iri (ϕi ) the i-th Fitting ring of F• and we will denote it

Hi (F•) := Ker(ϕi )/ Im(ϕi+1).

We call F• acyclic, if all Hi (F•) = 0 for i > 0. In that case, F• yields a finite free resolution of H0(F•). In case C is a Z-affine algebra with Z a local ring, we say that F• has Z- complexity at most c, if its length s is at most c, if all ai ≤ c, and if every entry of each ϕi has Z-complexity at most c. Below we will say that an element τ in a homology module Hi (F•) has Z-complexity at most c, if it is the image of a tuple in Ker(ϕi ) of Z-complexity at most c (for more details, see Section 11 below). Theorem 8.1 (Asymptotic Improved New Intersection Theorem). For each c, there exists a bound INIT(c) with the following property. Let V be a mixed characteristic discrete valuation ring and let (C, m) be a local V -affine domain. Let F• be a finite free complex over C. Assume H0(F•) has a minimal generator τ, such that Cτ has finite length and assume that c simultaneously bounds the V -complexity of C, τ and F•, the parameter degree of each Fitting ring

ϕs, p ϕs−1, p mix mix as mix as−1 (Fp• ) 0 → (R p ) −−−→(R p ) −−−−→ ...

ϕ2, p ϕ1, p mix a1 mix a0 −−−→(R p ) −−−→(R p ) → 0 mix of length s and of Op -complexity at most c, such that the i-th Fitting ring mix

a ϕs a − ϕs−1 ϕ2 a ϕ1 a (F•) 0 → R s −−→R s 1 −−−→ ... −−→R 1 −−→R 0 → 0 is a ﬁnite free complex. Let M denote its zeroth homology and ﬁx some i. By Łos’

Theorem, Iri (ϕi, p) is an O-approximation of Iri (ϕi ). By the uniform boundedness of the parameter degrees,

Iri (ϕi, p) and to the geometric height of Iri (ϕi ), by Theorem 5.13. In particular, by assumption, i ≤ d − di , and therefore, by deﬁnition of geometric height, we can

ﬁnd a generic sequence xi in R whose ﬁrst i entries belong to Iri (ϕi ). ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 455

:= Let B Ꮾ(R). Since xi is B-regular by Theorem 7.2, the grade of Iri (ϕi )B is at least i. Since this holds for all i, the Buchsbaum–Eisenbud–Northcott Acyclicity Theorem ([Bruns and Herzog 1993, Theorem 9.1.6]) proves that F•⊗R B is acyclic. Since B has depth at least d, it follows from Theorem 9.1.2 of the same reference that the zeroth homology of F• ⊗R B, that is to say, M ⊗R B, has depth at least d − s. Let τ be the ultraproduct of the τ p. Note that each τ p is by assumption the mix a0 mix image of a tuple in (R p ) of Op -complexity at most c, so that τ is already deﬁned over R by Lemma 2.1. By Łos’ Theorem, τ is a minimal generator of

mix mix H0(F• ⊗ LO (R)) = M ⊗ LO (R), and by [Schoutens 1999, Proposition 1.1] or [Jensen and Lenzing 1989, Proposition mix ∈ − 9.1], the length of LO (R)τ is at most c. By faithful ﬂatness, τ M mM, where m is the maximal ideal of R, and Rτ has length at most c. In particular, the image of τ ⊗ 1 in M/mM ⊗ B/mB is nonzero, and therefore τ ⊗ 1 itself is a nonzero element of M ⊗ B. Since mc annihilates τ ⊗ 1, we conclude that M ⊗ B has depth zero. Together with the conclusion from the previous paragraph, we get d ≤ s, contradiction. This type of argument by reductio ad absurdum, to obtain uniform bounds via ultraproducts, is very common and will be used constantly in the sequel. We will shorten the argument by saying from the start that by way of contradiction, we may assume that for some c, there exist for almost each p a counterexample with such and such properties.

9. Monomial and Direct Summand Conjectures

We keep notation as in the previous section, so that in particular O will denote the mix ultraproduct of mixed characteristic complete discrete valuation rings Op . In order to formulate a nonstandard version of the Monomial Conjecture, we need some terminology. Let ގ∞ be the ultrapower of ގ. Let Cw be rings, X := (X1,..., Xd ) indeterminates and A∞ the ultraproduct of the Cw[X]. Although each Cw[X] is ގ-graded, it is not true that A∞ is ގ∞-graded, since we might have inﬁnite sums d ν of monomials in A∞. Nonetheless, for each ν∞ ∈ (ގ∞) , the element X ∞ is well-deﬁned, namely, if ν∞ is the ultraproduct of elements νw ∈ ގ, then X ν∞ := ulim X ν p . w→∞

In particular, if B∞ is an arbitrary ultraproduct of rings Bw and if x is a d-tuple in ν B∞, then x ∞ is a well-defined element of B∞. d d By a cone H in a semigroup 0 (e.g., 0 = ގ or 0 = ގ∞), we mean a subset H of 0 such that ν +0 ⊆ H, for every ν ∈ H, where ν +0 stands for the collection of 456 HANS SCHOUTENS all ν + γ with γ ∈ 0. A cone H is finitely generated, if there exist ν1, . . . , νs ∈ H, called generators of the cone, such that [ H = νi + 0. i d If H is a cone in ގ , we let JH be the monomial ideal in ޚ[Y ] generated by all Y ν with ν ∈ H, where Y is a d-tuple of indeterminates. If H is generated by ν ν ν1, . . . , νs, then JH is generated by X 1 ,..., X s . Conversely, if J is a monomial ideal in ޚ[X], then the collection of all ν for which X ν ∈ J is a cone in ގd . Since ޚ[Y ] is noetherian, every cone in ގd is finitely generated. This is no longer true d for a cone in ގ∞. Let B be an arbitrary ring. We will use the following well-known fact about regular sequences. If x is a B-regular sequence (in fact, it suffices that x is quasireg- d ν ular), H a cone in ގ and ν∈ / H, then x does not lie in the ideal JH (x) generated by all xθ with θ ∈ H. Corollary 9.1. Let R be a local O-affine domain with equicharacteristic nonstan- eq d dard O-hull LO (R). Let x be a generic sequence in R, let H be a cone in ގ∞ and d let ν ∈ ގ∞. If ν∈ / H, then ν ∈ µ | ∈ eq (7) x / (x µ H)LO (R). d Proof. Suppose (7) is false for some choice of cone H of ގ∞ and some ν0 ∈/ H. eq In other words, we can find fi ∞ in LO (R) and tuples νi in H, such that

ν0 ν1 νs (8) x = f1∞x + · · · + fs∞x . → eq Since R Ꮾ(R) factors through LO (R), we can view (8) as a relation in Ꮾ(R), eq and we want to show that that is impossible. Let R p be an equicharacteristic O- eq + approximation of R, so that Ꮾ(R) is the ultraproduct of the (R p ) . Choose tuples eq + eq νi p ∈ ގ, elements fi p ∈ (R p ) and tuples xp in R p whose respective ultraproducts are νi , fi ∞ and x. By Łos’ Theorem, we get

ν0 p ν1 p νs p (9) xp = f1 pxp + · · · + fs pxp eq + in (R p ) , for almost all p. Łos’ Theorem also yields that ν0 p does not lie in the d cone of ގ generated by ν1 p, . . . , νs p, for almost all p. However, x is Ꮾ(R)-regular eq + by Theorem 7.2, whence, almost all xp are (R p ) -regular by Łos’ Theorem. By our above discussion on regular sequences, (9) cannot hold for those p. Theorem 9.2 (Asymptotic Monomial Conjecture I). For each c, there exists a bound MC(c) with the following property. Let Y be a tuple of indeterminates and J a monomial ideal in ޚ[Y ]. Let V be a mixed characteristic discrete valuation ring and let C be a local V -affine domain. Let y be a system of parameters in C ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 457 and let J(y)C denote the ideal in C obtained from J by the substitution Y 7→ y. Assume J V [Y ], C and y have V -complexity at most c and π c ∈ yC. If Y ν is a monomial of degree at most c not belonging to J, then yν ∈/ J(y)C, provided the characteristic of the residue field of V is bigger than MC(c). Proof. Note that since C has V -complexity at most c, its dimension d is at most c. By faithful flat descent, we may reduce to the case that V is complete. Suppose the result is false for some c, so that we can find for almost each prime number p,

• mix a mixed characteristic complete discrete valuation ring Op with uniformizing parameter π p, whose residue ﬁeld has characteristic p, • mix mix mix a local Op -afﬁne domain R p of Op -complexity at most c,

• multi-indices ν0 p, . . . , νt p such that νi p ≤ c and ν0 p is not in the cone generated by the remaining tuples, • mix a system of parameters yp of Op -complexity at most c generating an ideal c containing (π p) , such that

ν0 p ν1 p νt p mix (10) yp ∈ (yp ,..., yp )R p .

Note that the possible number t of tuples νi p is bounded in terms of c and hence can mix be taken to be independent of p. Let O be the ultraproduct of the Op and let R mix mix and LO (R) be the respective protoproduct and ultraproduct of the R p . Since R is then a domain, it is isodimensional. Let y and νi be the respective ultraproducts d d of yp and νi p. In particular, |νi | ≤ c, so that νi ∈ ގ . Let H be the cone in ގ generated by ν1, . . . , νt . By Łos’ Theorem, ν0 ∈/ H. The sequence y is deﬁned over R, by Lemma 2.1, and is generic in R, by Corollary 5.8. By an application of Łos’ Theorem to (10) together with Theorem 4.2, we get

yν0 ∈ (yν1 ,..., yνt )R.

However, this contradicts Corollary 9.1 for the cone H. Remark 9.3. In [Schoutens 2003b, Theorem 1.1], this result was stated erro- neously without imposing a bound on the degrees of the monomials. I can only prove this more general result in the special case given by Corollary 9.5 below. Using some results from [Schoutens ≥ 2007], we can remove the restriction on C to be a domain. Namely, by the usual argument, we reduce to the domain case by killing a minimal prime p of C of maximal dimension (that is to say, so that dim C = dim C/p). However, in order to apply the theorem to the domain C/p, we must be guaranteed that its V -complexity is at most c0, for some c0 only depending on c. Such a bound does indeed exist. 458 HANS SCHOUTENS

Theorem 9.4 (Asymptotic Direct Summand Conjecture). For each c, we can find a bound DS(c) with the following property. Let V be a mixed characteristic discrete valuation ring and let C → D be a finite, injective local V -algebra homomorphism of V -complexity at most c. If C is regular, then C is a direct summand of D (as a C-module), provided the characteristic of the residue field of V is bigger than the bound DS(c). Proof. If πC = 0, we are in the equicharacteristic case and the result is well-known. So we may assume that V ⊆ C. We leave it to the reader to make the reduction to the case that V is complete and D is torsion-free over V . Towards a contradiction, suppose for some c and almost each p, we have a mixed characteristic complete mix discrete valuation ring Op with residue field of characteristic p, and a finite, mix mix mix mix injective local Op -algebra homomorphism R p → S p of Op -complexity at mix mix most c, such that R p is regular but not a direct summand of S p . By the transfer described in Section 6.9, these data in mixed characteristic yield corresponding data in equal characteristic. In particular, we have for almost each p, eq an equicharacteristic p complete discrete valuation ring Op , and a finite, injective eq eq eq eq local Op -algebra homomorphism R p → S p of Op -complexity at most c, such eq that R p is regular. Although, we did not discuss transfer of homomorphisms and eq their properties, it is not hard to see, using faithfully flat descent, that almost no R p eq is a direct summand of S p . However, this is in violation of the Direct Summand theorem in equicharacteristic. Corollary 9.5 (Asymptotic Monomial Conjecture II). For each c, we can find a bound MC’(c) with the following property. Let V be a mixed characteristic discrete valuation ring, let D be a local V -affine algebra and let (x1,..., xd ) be a system of parameters in D. If there exists a finite, injective local V -algebra homomorphism C ⊆ D of V - complexity at most c, such that the xi belong to C and generate its maximal ideal, ··· t t+1 t+1 ≥ then (x1 xd ) does not belong to (x1 ,..., xd )D, for all t 0, provided the residue field of V is bigger than MC’(c) . Proof. We may take MC’(c) equal to the bound DS(c) from Theorem 9.4. Indeed, since D has dimension d, so does C, showing that C is regular. Hence C is a direct summand of D by Theorem 9.4, so that we are done by [Bruns and Herzog 1993, Lemma 9.2.2]. Note that the bounds provided by Theorem 9.2 for the problem at hand depend a priori also on the exponent t, so that the corollary gives a stronger result. Inter- estingly, by Cohen’s Structure Theorem, any system of parameters in a complete local V -affine domain arises as the image of a regular system of parameters under a finite extension. However, since we are forced to work with noncomplete V -affine algebras, it is not clear yet to which extent the above theorem applies. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 459

10. Pure subrings of regular rings

We keep notation as in the previous section, so that in particular O will denote mix the ultraproduct of mixed characteristic complete discrete valuation rings Op . Our goal is to show an asymptotic version of the Hochster–Roberts Theorem in [Hochster and Roberts 1974]. Recall that a ring homomorphism C → D is called cyclically pure if every ideal I in C is extended from D, that is to say, if I = ID∩C.

Theorem 10.1. If R is a pseudoregular isodimensional local O-affine algebra, then R → Ꮾ(R) is faithfully flat. Proof. Let L be a linear form in a finite number of indeterminates Y with co- eq eq efficients in R and let b be a solution in B := Ꮾ(R) of L = 0. Let R p , L p eq and bp be equicharacteristic O-approximations of R, L and b respectively. By eq eq + eq Łos’ Theorem, bp is a solution in (R p ) of the linear equation L p = 0. By eq eq eq [Aschenbrenner 2001a, Corollary 4.27], we can find tuples a1 p ,..., as p over R p eq eq generating the module of solutions of L p = 0, all of Op -complexity at most c, for some c independent from p and s. Let a1,..., as be the respective ultraproducts, which are then defined over R by Lemma 2.1. By Łos’ Theorem, L(ai ) = 0, for eq each i. On the other hand, almost all R p are regular, by Theorem 6.5. Therefore, eq eq + eq R p → (R p ) is flat by [Huneke 1996, Theorem 9.1]. Hence we can write bp as eq + eq a linear combination over (R p ) of the ai p . By Łos’ Theorem, b is a B-linear combination of the solutions ai , showing that R → B is flat whence faithfully flat. Proposition 10.2. Let R → S be an injective homomorphism of local isodimensional O-affine algebras. If R/π R → S/π S is cyclically pure and S is a pseudoregular local ring, then R is pseudo-Cohen–Macaulay.

Proof. Since S is a domain by Corollary 6.6, so is R. If π R = 0, we are in an equicharacteristic noetherian situation and the statement becomes the Hochster– Roberts Theorem [Hochster and Roberts 1974]. Therefore, we may assume π is R-regular, so that we can choose a generic sequence x := (x1,..., xd ) in R with x1 = π. For each n ≤ d, let In := (x1,..., xn)R. Suppose rxn+1 ∈ In, for some r ∈ R. By Theorem 7.2, the sequence x is a Ꮾ(R)-regular. Therefore, r ∈ InᏮ(R). Since the homomorphism R → S induces a homomorphism Ꮾ(R) → Ꮾ(S), we get r ∈ InᏮ(S). By Theorem 10.1, we have

InᏮ(S) ∩ S = In S, so r ∈ In S. Using ﬁnally that R/π R → S/π S is cyclically pure and π ∈ In, we get r ∈ In. This shows that x is R-regular, so that R has depth at least d and hence is pseudo-Cohen–Macaulay. 460 HANS SCHOUTENS

Theorem 10.3 (Asymptotic Hochster–Roberts Theorem). For each c, we can find a bound HR(c) with the following property. Let V be a mixed characteristic discrete valuation ring and let C → D be a local V -algebra homomorphism of V - complexity at most c. If C → D is cyclically pure and D is regular, then C is Coh- en–Macaulay, provided the characteristic of the residue field of V is at least HR(c). Proof. As before, we may reduce to the case that V is complete and that V ⊆ C. Suppose this assertion is then false for some c, so that we can find for almost each mix prime number p, a mixed characteristic complete discrete valuation ring Op with mix residue field of characteristic p and a cyclically pure Op -algebra homomorphism mix mix mix mix mix R p → S p of Op -complexity at most c, such that S p is regular but R p is → mix → mix not Cohen–Macaulay. Let R S and LO (R) LO (S) be respectively the mix mix protoproduct and the ultraproduct of the R p → S p . Theorem 6.3 implies that R is not pseudo-Cohen–Macaulay, and Theorem 6.5, that S is pseudoregular. I claim that R/π R → S/π S is cyclically pure. Assuming this claim, we get from Proposition 10.2 that R is pseudo-Cohen–Macaulay, contradiction. To prove the claim, let I be an arbitrary ideal in R containing π. Let r ∈ IS ∩ R, so that we need to show that r ∈ I . Note that I is finitely generated, as R/π R is mix mix mix noetherian. Let I p and r p be mixed characteristic O-approximations in R p mix mix mix mix of I and r respectively. By Łos’ Theorem, almost all r p lie in I p S p ∩ R p , mix ∈ mix ∈ whence in I p by cyclical purity. By Łos’ Theorem, r I LO (R), so that r I by faithful flatness, as we needed to prove.

11. Asymptotic vanishing for maps of Tor

Proposition 11.1. If R → S is an integral extension of local O-affine domains, then Ꮾ(R) = Ꮾ(S). Proof. Since any integral extension is a direct limit of finite extensions, we may assume that R → S is finite. Choose an equicharacteristic O-approximation

eq eq R p → S p eq eq of R → S. By Theorem 4.4 and Łos’ Theorem, almost all R p and S p are domains eq eq eq + eq + and the extension R p → S p is ﬁnite. Therefore, (R p ) = (S p ) , so that in the ultraproduct, we get Ꮾ(R) = Ꮾ(S). Theorem 11.2. Let R → S → T be local O-algebra homomorphisms between local O-afﬁne domains. Assume that R and T are pseudoregular and that R → S is integral and injective. For every R-module M, the induced map R R Tori (S, M) → Tori (T, M) is zero for all i ≥ 1. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 461

Proof. Since R → S is integral, we have that Ꮾ(R) = Ꮾ(S) by Proposition 11.1. R Therefore, Tori (Ꮾ(S), M) = 0, for all i ≥ 1, by Theorem 10.1. By weak functoriality, we have, for each i ≥ 1, a commutative diagram

R - R Tori (S, M) Tori (T, M)

(11) ? ? R - R 0 = Tori (Ꮾ(S), M) Tori (Ꮾ(T ), M).

In particular, the composite map in this diagram is zero, so that the statement follows once we have shown that the last vertical map is injective. However, this is clear, since T → Ꮾ(T ) is faithfully flat by Theorem 10.1. To make use of this theorem, we need to incorporate modules in our present setup. I will not provide full details, since many results are completely analogous to the case where we work over a field, and this has been treated in detail in [Schoutens 2000a]. Of course, we do not have the full equivalent of Theorem 2.2 to our disposal, but for most purposes, the flatness result in Theorem 4.2 suffices. Let C be an arbitrary noetherian local ring and M a finitely generated module over C. We say that a finite free complex F• is a finite free resolution of M up to level n, if H0(F•) = M and all H j (F•) = 0, for j = 1,..., n. Hence, if n is strictly larger than the length of F•, then this just means that F• is a finite free resolution of M (compare with the terminology introduced in the beginning of Section 8). Suppose moreover that Z is a noetherian local ring and C is a local Z-affine algebra. We say that M has Z-complexity at most c, if C has Z-complexity at most c and if M can be realized as the cokernel of a matrix of Z-complexity at most c (meaning that its size is at most c and all its entries have Z-complexity at most c). Proposition 11.3. For each pair (c, n), there exist bounds RES(c, n) and HOM(c) with the following property. Let V be a mixed characteristic discrete valuation ring and let C be a local V -affine algebra of V -complexity at most c. • Any finitely generated C-module of V -complexity at most c, admits a (minimal) finite free resolution up to level n of V -complexity at most RES(c, n). • Any finite free complex over C of V -complexity at most c, has homology modules of V -complexity at most HOM(c). Proof. The first assertion follows by induction from the already quoted [Aschen- brenner 2001a, Corollary 4.27] on bounds of syzygies (compare with the proof of [Schoutens 2000a, Theorem 4.3]). It is also clear that we may take this resolution to be minimal (=every tuple in one of the kernels has its entries in the maximal ideal), if we choose to do so. The second assertion is derived from the flatness of 462 HANS SCHOUTENS the nonstandard O-hull in exactly the same manner as the corresponding result for fields was obtained in [Schoutens 2000a, Lemma 4.2 and Theorem 4.3]. Recall that the weak global dimension of a ring C is by definition the supremum (possibly infinite) of the weak homological dimensions (=flat dimensions) of all C- C modules, that is to say, the supremum of all n for which Torn (·, ·) is not identically zero.

Corollary 11.4. A pseudoregular local O-afﬁne domain has ﬁnite weak global dimension.

Proof. Let R be a pseudoregular local O-affine domain. Given an arbitrary R- module M, we have to show that M has bounded flat dimension, that is to say, admits a flat resolution of bounded length. Assume first that M is finitely presented. Hence we can realize M as the cokernel of some matrix 0. Let L(R) be the nonstandard O-hull of R and let Rw and 0w be O-approximations of R and 0 respectively. Let Mw be the cokernel of 0w. Let d be the geometric dimension of R. By Proposition 11.3, we can find a finite free resolution Fw• up to level d of each Mw, of Ow-complexity at most c, for some c depending only on 0, whence independent from w. Since almost each Rw is regular by Theorem 6.5 and has dimension d by Theorem 5.4, almost each Mw has projective dimension at most d, so that we can even assume that Fw• is a finite free resolution of Mw. Let F• be the protoproduct of the Fw• (that is to say, the finite free complex over R given by the protoproduct of the matrices in Fw•). By Łos’ Theorem, F• ⊗R L(R) is a free resolution of M ⊗R L(R), and therefore by faithful flat descent, F• is a free resolution of M, proving that M has projective dimension at most d. Assume now that M is arbitrary. By what we just proved, we have for every R finitely generated ideal I of R that Tord+1(M, R/I ) vanishes. Hence, if H is a d-th R syzygy of M, then Tor1 (H, R/I ) = 0. Since this holds for every finitely generated ideal of R, we get from [Matsumura 1986, Theorem 7.7] that H is flat over R. Hence M has finite flat dimension at most d. By [Jensen 1970], any flat R-module has projective dimension less than the finitistic global dimension of R (the supremum of all projective dimensions of modules of finite projective dimension). Therefore, if, moreover, the finitistic global dimension of R is finite, then so is its global dimension. For a noetherian local ring, its global dimension is finite if and only if its residue field has finite projective dimension (if and only if it is regular). Here is the pseudo analogue of this.

Corollary 11.5. A local O-affine domain is pseudoregular if and only if it is a coherent regular ring in the sense of [Bertin 1971], if and only if its residue field has finite projective dimension. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 463

Proof. In [Bertin 1971] or [Glaz 1992, §5], a local ring R is called a coherent regular ring, if every finitely generated ideal of R has finite projective dimension. If R is a pseudoregular local O-affine domain, then this property was established in the course of the proof of Corollary 11.4. Conversely, suppose R is a local O-affine domain in which every finitely generated ideal has finite projective dimension. In particular, its residue field k admits a finite projective resolution, say of length n. Let Rw and kw be O-approximations of R and k respectively. Since the kw have uniformly bounded Ow-complexity, Proposition 11.3 allows us to take a minimal finite free resolution Fw• of kw up to level n, with the property that each Fw• has Ow-complexity at most c, for some c independent from w. Let F• be the protoproduct of these resolutions. By Łos’ Theorem and faithfully flat descent, F• is a minimal finite free resolution of k up to level n. Since F• is minimal and since k has by assumption projective dimension n, it follows that the final morphism (that is to say, the left most arrow) in F• is injective. By Łos’ Theorem, so are almost all final morphisms in Fw•, showing that almost all kw have finite projective dimension. By Serre’s characterization of regular local rings, we conclude that almost all Rw are regular. Theorem 6.5 then yields that R is pseudoregular, as we wanted to show. Closer inspection of the above argument shows that the residue field of a pseudoregular local O-affine domain R has projective dimension equal to the geometric dimension of R. In particular, the weak global dimension of R is equal to its geometric dimension.

Theorem 11.6 (Asymptotic Vanishing for Maps of Tors). For each c, we can find a bound VT(c) with the following property. Let V be a mixed characteristic discrete valuation ring, let C → D → E be local V -algebra homomorphisms of local V - affine domains and let M be a finitely generated R-module, all of V -complexity at most c. If C and E are regular and C → D is finite and injective, then the natural map C C Torn (D, M) → Torn (E, M) is zero, for all n ≥ 1, provided the characteristic of the residue field of V is at least VT(c).

C Proof. Note that C has dimension at most c and therefore Torn (·, ·) vanishes identically for all n > c and the assertion trivially holds for these values of n. If πC = 0, we are in the equicharacteristic case, for which the result is known ([Huneke 1996, Theorem 9.7]). Hence we may assume that all rings are torsion-free over V . More- over, without loss of generality, we may assume that V is complete. Suppose even in this restricted setting, there is no such bound for c and some 1 ≤ n ≤ c. Hence, for almost each prime number p, we can ﬁnd a counterexample consisting of the following data: 464 HANS SCHOUTENS

• mix a mixed characteristic complete discrete valuation ring Op of residual characteristic p; • mix mix mix mix mix local R p -algebra homomorphisms R p → S p → T p of Op -complexity mix mix at most c between torsion-free local domains, with R p and T p regular and mix mix R p → S p finite and injective; • mix mix mix a finitely generated R p -module M p of Op -complexity at most c; such that mix mix R p mix mix R p mix mix Torn (S p , M p ) → Torn (T p , M p ) is nonzero. mix mix Let O be the ultraproduct of the Op and let M be the protoproduct of the M p (that is to say, M is the cokernel of the protoproduct of matrices whose cokernel mix → → mix → mix → mix is M p ). Let R S T and LO (R) LO (S) LO (T ) be the respective mix protoproduct and mixed characteristic ultraproduct of the homomorphisms R p → mix mix S p → T p . It follows from Corollary 6.6 and Theorems 4.4 and 6.5, that R, S and T are local O-affine domains with R and T pseudoregular. By Łos’ Theorem, mix → mix mix mix → mix using that the R p S p have bounded Op -complexity, LO (R) LO (S) is finite, whence so is R → S by faithful flat descent. By Theorem 11.2, the natural R R homomorphism Torn (S, M) → Torn (T, M) is therefore zero. mix mix By Proposition 11.3, we can find a finite free resolution Fp• of M p up to mix 0 0 level n, of Op complexity at most c , for some c only depending on c (note that n ≤ c). By definition of Tor, we have isomorphisms

mix R p mix mix ∼ mix mix Tor (S , M ) = H (F ⊗ mix S ) n p p n p• R p p mix R p mix mix ∼ mix mix Tor (T , M ) = H (F ⊗ mix T ) n p p n p• R p p mix 00 In particular, by Proposition 11.3, both modules have Op -complexity at most c , 00 0 for some c only depending on c , whence only on c. Let HS and HT be their respective protoproduct, so that by Łos’ Theorem and our assumptions, HS → HT mix is nonzero. Let F• be the protoproduct of the Fp• . By Łos’ Theorem and faithful flatness, HS and HT are isomorphic to Hn(F• ⊗R S) and Hn(F• ⊗R T ) respectively. Since F• is a finite free resolution of M up to level n by another application of Łos’ Theorem and faithful flatness, these two modules are also isomorphic to R R Torn (S, M) and Torn (T, M) respectively. Hence the natural map between these two modules is nonzero, contradiction.

References

[Aschenbrenner 2001a] M. Aschenbrenner, Ideal membership in polynomial rings over the integers, Ph.D. thesis, University of Illinois, Urbana-Champaign, 2001. ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 465

[Aschenbrenner 2001b] M. Aschenbrenner, “Remarks on a paper by Sabbagh”, manuscript, 2001. [Aschenbrenner and Schoutens 2007] M. Aschenbrenner and H. Schoutens, “Lefschetz extensions, tight closure and big Cohen–Macaulay algebras”, Israel J. Math. (2007). To appear. [Ax and Kochen 1965] J. Ax and S. Kochen, “Diophantine problems over local fields, I, II”, Amer. J. Math. 87 (1965), 605–648. MR 32 #2401 Zbl 0136.32805 [Becker et al. 1979] J. Becker, J. Denef, L. Lipshitz, and L. van den Dries, “Ultraproducts and approximations in local rings, I”, Invent. Math. 51:2 (1979), 189–203. MR 80k:14009 Zbl 0416.13004 [Bertin 1971] J. Bertin, “Anneaux cohérents réguliers”, C. R. Acad. Sci. Paris Sér. A-B 273 (1971), A1–A2. MR 43 #4820 Zbl 0216.32502 [Bruns and Herzog 1993] W. Bruns and J. Herzog, Cohen–Macaulay rings, Cambridge Studies in Advanced Mathematics 39, Cambridge University Press, Cambridge, 1993. MR 95h:13020 Zbl 0788.13005 [van den Dries 1979] L. van den Dries, “Algorithms and bounds for polynomial rings”, pp. 147–157 in Logic Colloquium ’78 (Mons, 1978), edited by M. Boffa et al., Stud. Logic Foundations Math. 97, North-Holland, Amsterdam, 1979. MR 81f:03045 Zbl 0461.13015 [van den Dries and Schmidt 1984] L. van den Dries and K. Schmidt, “Bounds in the theory of polynomial rings over fields: a nonstandard approach”, Invent. Math. 76:1 (1984), 77–91. MR 85i:12016 Zbl 0539.13011 [Eisenbud 1995] D. Eisenbud, Commutative algebra with a view toward algebraic geometry, Grad- uate Texts in Mathematics 150, Springer, New York, 1995. MR 97a:13001 Zbl 0819.13001 [Ershov 1965] Y. L. Ershov, “On the elementary theory of maximal normed fields”, Algebra i Logika Sem. 4:3 (1965), 31–70. In Russian. MR 33 #1307 [Ershov 1966] Y. L. Ershov, “On the elementary theory of maximal normed fields, II”, Algebra i Logika Sem. 5:1 (1966), 5–40. In Russian. MR 34 #4134 [Evans and Griffith 1981] E. G. Evans and P. Griffith, “The syzygy problem”, Ann. of Math. (2) 114:2 (1981), 323–333. MR 83i:13006 Zbl 0497.13013 [Glaz 1992] S. Glaz, “Commutative coherent rings: historical perspective and current develop- ments”, Nieuw Arch. Wisk. (4) 10:1-2 (1992), 37–56. MR 93j:13024 Zbl 0787.13001 [Heitmann 2002] R. C. Heitmann, “The direct summand conjecture in dimension three”, Ann. of Math. (2) 156:2 (2002), 695–712. MR 2003m:13008 Zbl 1076.13511 [Hochster 1975a] M. Hochster, “Big Cohen–Macaulay modules and algebras and embeddability in rings of Witt vectors”, pp. 106–195 in Conference on Commutative Algebra (Kingston, Ont., 1975), edited by A. V. Geramita, Queen’s Papers on Pure and Applied Math. 42, Queen’s Univ., Kingston, Ont., 1975. MR 53 #407 Zbl 0342.13009 [Hochster 1975b] M. Hochster, Topics in the homological theory of modules over commutative rings, CBMS Regional Conference Series in Math. 24, Amer. Math. Soc., Providence, RI, 1975. MR 51 #8096 Zbl 0302.13003 [Hochster 1983] M. Hochster, “Canonical elements in local cohomology modules and the direct summand conjecture”, J. Algebra 84:2 (1983), 503–553. MR 85j:13021 Zbl 0562.13012 [Hochster 1994] M. Hochster, “Solid closure”, pp. 103–172 in Commutative algebra: syzygies, mul- tiplicities, and birational algebra (South Hadley, MA, 1992), edited by C. Huneke et al., Contemp. Math. 159, Amer. Math. Soc., Providence, RI, 1994. MR 95a:13011 Zbl 0812.13006 [Hochster 2002] M. Hochster, “Big Cohen-Macaulay algebras in dimension three via Heitmann’s theorem”, J. Algebra 254:2 (2002), 395–408. MR 2004c:13011 Zbl 1078.13506 466 HANS SCHOUTENS

[Hochster and Huneke 1989] M. Hochster and C. Huneke, “Tight closure”, pp. 305–324 in Com- mutative algebra (Berkeley, 1987), edited by M. Hochster et al., Math. Sci. Res. Inst. Publ. 15, Springer, New York, 1989. MR 91f:13022 Zbl 0741.13002 [Hochster and Huneke 1992] M. Hochster and C. Huneke, “Infinite integral extensions and big Cohen-Macaulay algebras”, Ann. of Math. (2) 135 (1992), 53–89. MR 92m:13023 Zbl 0753.13003 [Hochster and Huneke 2000] M. Hochster and C. Huneke, “Tight closure in equal characteristic zero”, preprint, 2000, see http://www.math.lsa.umich.edu/~hochster/tcz.ps.Z. [Hochster and Roberts 1974] M. Hochster and J. L. Roberts, “Rings of invariants of reductive groups acting on regular rings are Cohen-Macaulay”, Advances in Math. 13 (1974), 115–175. MR 50 #311 Zbl 0289.14010 [Hodges 1993] W. Hodges, Model theory, Encyclopedia Mathematics and its Applications 42, Cam- bridge University Press, Cambridge, 1993. MR 94e:03002 Zbl 0789.03031 [Huneke 1996] C. Huneke, Tight closure and its applications, CBMS Regional Conference Series in Mathematics 88, Amer. Math. Soc., Providence, RI, 1996. MR 96m:13001 Zbl 0930.13004 (i) [Jensen 1970] C. U. Jensen, “On the vanishing of lim←− ”, J. Algebra 15 (1970), 151–166. MR 41 #5460 Zbl 0199.36202 [Jensen and Lenzing 1989] C. U. Jensen and H. Lenzing, Model-theoretic algebra with particular emphasis on fields, rings, modules, Algebra, Logic and Applications 2, Gordon and Breach Science Publishers, New York, 1989. MR 91m:03038 Zbl 0728.03026 [Matsumura 1986] H. Matsumura, Commutative ring theory, Cambridge Studies in Advanced Math- ematics 8, Cambridge University Press, Cambridge, 1986. Translated from the Japanese by M. Reid. MR 88h:13001 Zbl 0603.13001 [Peskine and Szpiro 1973] C. Peskine and L. Szpiro, “Dimension projective finie et cohomologie lo- cale: applications à la démonstration de conjectures de M. Auslander, H. Bass et A. Grothendieck”, Inst. Hautes Études Sci. Publ. Math. 42 (1973), 47–119. MR 51 #10330 Zbl 0268.13008 [Raynaud and Gruson 1971] M. Raynaud and L. Gruson, “Critères de platitude et de projectiv- ité: Techniques de “platification” d’un module”, Invent. Math. 13 (1971), 1–89. MR 46 #7219 Zbl 0227.14010 [Roberts 1987] P. Roberts, “Le théorème d’intersection”, C. R. Acad. Sci. Paris Sér. I Math. 304:7 (1987), 177–180. MR 89b:14008 Zbl 0609.13010 [Sabbagh 1974] G. Sabbagh, “Coherence of polynomial rings and bounds in polynomial ideals”, J. Algebra 31 (1974), 499–507. MR 50 #297 Zbl 0292.13007 [Schmidt-Göttsch 1987] K. Schmidt-Göttsch, “Bounds and definability over fields”, J. Reine Angew. Math. 377 (1987), 18–39. MR 89b:12012 Zbl 0605.13015 [Schoutens 1999] H. Schoutens, “Existentially closed models of the theory of Artinian local rings”, J. Symbolic Logic 64:2 (1999), 825–845. MR 2001g:03068 Zbl 1060.03056 [Schoutens 2000a] H. Schoutens, “Bounds in cohomology”, Israel J. Math. 116 (2000), 125–169. MR 2001c:13038 Zbl 0953.13005 [Schoutens 2000b] H. Schoutens, “Uniform bounds in algebraic geometry and commutative algebra”, pp. 43–93 in Connections between model theory and algebraic and analytic geometry, edited by A. Macintyre, Quad. Mat. 6, Aracne, Roma, 2000. MR 2003g:13029 Zbl 01679771 [Schoutens 2003a] H. Schoutens, “Lefschetz principle applied to symbolic powers”, J. Algebra Appl. 2:2 (2003), 177–187. MR 2004c:13040 Zbl 1095.13031 [Schoutens 2003b] H. Schoutens, “Mixed characteristic homological theorems in low degrees”, C. R. Math. Acad. Sci. Paris 336:6 (2003), 463–466. MR 2004d:13021 Zbl 1049.13009 ASYMPTOTIC HOMOLOGICAL CONJECTURES IN MIXED CHARACTERISTIC 467

[Schoutens 2003c] H. Schoutens, “A non-standard proof of the Briançon-Skoda theorem”, Proc. Amer. Math. Soc. 131:1 (2003), 103–112. MR 2003i:13004 Zbl 1005.13007 [Schoutens 2003d] H. Schoutens, “Non-standard tight closure for afﬁne ރ-algebras”, Manuscripta Math. 111:3 (2003), 379–412. MR 2004m:13019 Zbl 1082.13005 [Schoutens 2004a] H. Schoutens, “Canonical big Cohen-Macaulay algebras and rational singularities”, Illinois J. Math. 48:1 (2004), 131–150. MR 2005e:13012 Zbl 1045.13005 [Schoutens 2004b] H. Schoutens, “Dimension and singularity theory for local rings of ﬁnite embedding dimension”, preprint, 2004, see http://websupport1.citytech.cuny.edu/Faculty/hschoutens/ PDF/DimFinEmb.pdf. [Schoutens 2005a] H. Schoutens, “Log-terminal singularities and vanishing theorems via non-standard tight closure”, J. Algebraic Geom. 14:2 (2005), 357–390. MR 2006e:13005 Zbl 1070.14005 [Schoutens 2005b] H. Schoutens, “Pure subrings of regular rings are pseudo-rational”, preprint, 2005. To appear in Trans. Amer. Math. Soc. math.AC/0507304 [Schoutens 2006] H. Schoutens, “Absolute bounds on the number of generators of Cohen–Macaulay ideals of height at most two”, Bull. Belgian Math. Soc. 13 (2006), 719–732. [Schoutens 2007] H. Schoutens, “Bounds in polynomial rings over Artinian local rings”, Monatsh. Math. (2007). To appear. [Schoutens ≥ 2007] H. Schoutens, Use of ultraproducts in commutative algebra, book in preparation. [Strooker 1990] J. R. Strooker, Homological questions in local algebra, London Math. Soc. Lecture Note Series 145, Cambridge University Press, Cambridge, 1990. MR 91m:13013 Zbl 0786.13008

Received October 25, 2003.

HANS SCHOUTENS DEPARTMENT OF MATHEMATICS CITY UNIVERSITYOF NEW YORK 365 FIFTH AVENUE NEW YORK, NY 10016 UNITED STATES [email protected] http://websupport1.citytech.cuny.edu/Faculty/hschoutens PACIFIC JOURNAL OF MATHEMATICS Vol. 230, No. 2, 2007

THE FIXED POINT SUBALGEBRA OF A LATTICE VERTEX OPERATOR ALGEBRA BY AN AUTOMORPHISM OF ORDER THREE

KENICHIRO TANABEAND HIROMICHI YAMADA

We study the subalgebra of the lattice vertex operator algebra V√ con- 2A2 sisting of the ﬁxed points of an automorphism which is induced from an order-three isometry of the root lattice A2. We classify the simple modules for the subalgebra. The rationality and the C2-coﬁniteness are also established.

1. Introduction

The space of fixed points of an automorphism group of finite order in a vertex operator algebra is a vertex operator subalgebra. The study of such subalgebras and their modules is called orbifold theory. It is a rich field both in conformal field theory and in the theory of vertex operator algebras. However, orbifold theory is difficult to study in general. One reason is that the subalgebra of fixed points usually has more complicated structure than the original vertex operator algebra. The first example of orbifold theory in vertex operator algebras is the moonshine module V \ by Frenkel, Lepowsky, and Meurman [Frenkel et al. 1988], constructed + T,+ + as an extension of V3 by its simple module V3 , where V3 is the space of fixed points of an automorphism θ of order two in the Leech lattice vertex operator algebra V3. This construction is called a 2B-orbifold construction because θ corresponds to a 2B involution of the monster simple group. More generally, Frenkel et al. defined a vertex operator algebra VL associated with an arbitrary positive definite even lattice L. These lattice vertex operator algebras provide a large family of vertex operator algebras. Such a lattice vertex operator algebra admits an automorphism θ of order two, which is a lift of the isometry α 7→ −α + of the underlying lattice L. Orbifold theory for the fixed point subalgebra VL of + θ has been developed extensively. The simple VL -modules have been classified

MSC2000: primary 17B69; secondary 17B68. Keywords: vertex operator algebra, orbifold, W3 algebra. Tanabe was partially supported by JSPS Grant-in-Aid for Scientiﬁc Research No. 17740002. Yamada was partially supported by JSPS Grant-in-Aid for Scientiﬁc Research No. 17540016.

469 470 KENICHIRO TANABE AND HIROMICHI YAMADA

[Abe and Dong 2004], the fusion rules have been determined [Abe et al. 2005], and + it has been established that VL is C2-cofinite [Abe et al. 2004; Yamskulna 2004]. Here we study the fixed point subalgebra by an automorphism√ of order√ three for a certain lattice vertex operator algebra. Namely, let L = 2A2 be 2 times an ordinary root lattice of type A2 and let τ be an isometry of the root lattice of type A2 induced from an order-three permutation on the set of positive roots. We τ classify the simple modules for the subalgebra VL of fixed points by τ. Moreover, τ we show that VL is rational and C2-cofinite. In [Dong et al. 2004; Kitazume et al. 2003] we have already discussed the vertex τ τ 0 0 operator algebra VL . It was shown that VL = M ⊕ W is a direct sum of a subalgebra M0 and its simple highest-weight module W 0. Actually, M0 is a tensor product of a W3 algebra of central charge 6/5 and a W3 algebra of central charge 4/5. The property of a W3 algebra of central charge 6/5 as the first component of the tensor product M0 was investigated in [Dong et al. 2004]. It is generated by the Virasoro element ω˜ 1 and a weight-three vector J. The second component of 0 M , a W3 algebra of central charge 4/5, was studied in [Kitazume et al. 2000b]. It is generated by the Virasoro element ω˜ 2 and a weight-three vector K . Each of these W3 algebras possesses a symmetry of order three. The order-three symmetry ⊥ ∼ of the second W3 algebra is related to the ޚ3 part of L /L = ޚ2 × ޚ2 × ޚ3, where L⊥ denotes the dual lattice of L. As an M0-module, W 0 is generated by a highest- τ weight vector P of weight 2. Thus the vertex operator algebra VL is generated by the five elements ω˜ 1, ω˜ 2, J, K , and P. There are 12 inequivalent simple VL -modules, which correspond to the cosets of ⊥ L in its dual lattice L [Dong 1993]. Let (U, YU ) be a simple VL -module. One can define a new simple VL -module (U ◦ τ, YU◦τ ) by U ◦ τ = U as vector spaces and YU◦τ (v, z) = YU (τv, z) for v ∈ VL . Then U 7→ U ◦τ is a permutation on the set of simple VL -modules. In the case where U and U ◦ τ are equivalent VL -modules, U ε is said to be τ-stable.√ If U is τ-stable, the eigenspace U(ε) of τ with eigenvalue ξ , τ where ξ = exp(2π −1/3), ε = 0, 1, 2, is a simple VL -module, while if U belongs τ to a τ-orbit of length three, U itself is a simple VL -module and the three members in the τ-orbit are equivalent [Dong and Yamskulna 2002, Theorem 6.14]. Among the 12 inequivalent simple VL -modules, three are τ-stable and the remaining nine τ are divided into three τ-orbits. In this way we obtain 12 simple VL -modules. It is known that there are three inequivalent simple τ-twisted VL -modules and three 2 inequivalent simple τ -twisted VL -modules. We denote them respectively by

Tχ T 0 j j j 2 χ j 2 (1-1) VL (τ) := VL (τ), VL (τ ) := VL (τ ), j = 0, 1, 2.

2 The automorphism τ acts on these τ-twisted or τ -twisted VL -modules and each τ eigenspace of τ is a simple VL -module [Miyamoto and Tanabe 2004, Theorem 2]. ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 471

τ There are 18 such simple VL -modules, all of them inequivalent. Hence there are τ at least 30 inequivalent simple VL -modules. τ The main part of our argument is to show that every simple VL -module is iso- τ τ 0 0 morphic to one of these 30 simple VL -modules. Recall that VL = M ⊕ W and 0 that M is a tensor product of two W3 algebras. The W3 algebra of central charge 6/5 (resp. 4/5) possesses 20 (resp. 6) inequivalent simple modules. Thus there are 120 inequivalent simple M0-modules. It turns out that among these simple M0- 0 τ modules, 60 of them cannot appear as an M -submodule in any simple VL -module τ and that each simple VL -module is a direct sum of two of the remaining 60 simple 0 0 0 τ M -modules. We note that W is not a simple current M -module. Thus VL is a nonsimple current extension of M0. A discussion on simple modules for another nonsimple current extension of a certain vertex operator algebra can be found in [Lam et al. 2005, Appendix C]. The organization of this paper is as follows. In Section 2 we review various notions about untwisted or twisted modules for vertex operator algebras, together with some basic tools which will be used in later sections. In Section 3 we ﬁx τ notation for the vertex operator algebra VL and collect its properties. We clarify 0 i 0 i an argument on the simplicity of MT (τ ) and WT (τ ), i = 1, 2, in [Kitazume et al. 2003, Proposition 6.8]. Furthermore, we correct some misprints in [Kitazume et al. 2003, (6.46)] and in an equation of [Dong et al. 2004, page 265] concerning a j decomposition of a simple τ-twisted VL -module VL (τ), j = 1, 2 as a τ-twisted 0 0 Mk ⊗ Mt -module (see Remark 3.5). In Section 4 we discuss the structure of the τ 1 30 known simple VL -modules. In particular, we calculate the action of o(ω˜ ), o(ω˜ 2), o(J), o(K ), and o(P) on the top level of these simple modules. Finally, in τ Section 5 we complete the classiﬁcation of simple VL -modules. We also show the τ rationality of VL . The authors would like to thank Ching Hung Lam, Masahiko Miyamoto, and Hiroshi Yamauchi for valuable discussions. The proof of Lemma 5.7 is essentially the same as that of [Lam et al. 2005, Lemma C.3]. Part of our calculation was done by a computer algebra system Risa/Asir. The authors are grateful to Kazuhiro Yokoyama for helpful advice on computer programs.

2. Preliminaries

We recall some notation for untwisted or twisted modules for a vertex operator algebra. We also review the twisted version of Zhu’s theory. A basic reference to twisted modules is [Dong et al. 1998a]. For untwisted modules, see also [Lep- owsky and Li 2004]. Let (V, Y, 1, ω) be a vertex operator algebra√ and g be an automorphism of V of ﬁnite order T . Set V r = {v ∈ V | gv = e2π −1r/T v}, so that L r V = r∈ޚ/T ޚ V . 472 KENICHIRO TANABE AND HIROMICHI YAMADA

Deﬁnition 2.1. A weak g-twisted V -module M is a vector space equipped with a linear map

X −n−1 YM ( · , z) : v ∈ V 7→ YM (v, z) = vn z ∈ (End M){z} n∈ޑ satisfying the following conditions. P −n−1 r (1) YM (v, z) = n∈r/T +ޚ vn z for v ∈ V .

(2) vnw = 0 if n 0, where v ∈ V and w ∈ M.

(3) YM (1, z) = idM . (4) For u ∈ V r and v ∈ V , the g-twisted Jacobi identity holds: − − −1 z1 z2 − −1 z2 z1 (2-1) z0 δ YM (u, z1)YM (v, z2) z0 δ YM (v, z2)YM (u, z1) z0 −z0 − −r/T − = −1 z1 z0 z1 z0 z2 δ YM (Y (u, z0)v, z2). z2 z2 −l−1 −m−1 −n−1 ∈ r Compare the coefﬁcients of z0 z1 z2 in both sides of (2-1) for u V , ∈ s ∈ ∈ r + ∈ s + v V , l ޚ, m T ޚ, and n T ޚ. Then we obtain ∞ ∞ X m X i l l (2-2) (u + v) + − = (−1) u + − v + − (−1) v + − u + . i l i m n i i l m i n i l n i m i i=0 i=0 In the case l = 0, this reduces to ∞ X m (2-3) [u , v ] = (u v) + − . m n i i m n i i=0

0 The Virasoro element ω is contained in V . Let L(n) = ωn+1 for n ∈ ޚ. Then [ ] = − + + 1 3 − L(m), L(n) (m n)L(m n) 12 (m m)δm+n,0(rankV ), d Y (v, z) = Y (L(−1)v, z) dz M M for v ∈ V ; see [Dong et al. 1998a, (3.8), (3.9)]. An important consequence of (2-1) is the associativity formula

k+r/T k+r/T (2-4) (z0+z2) YM (u,z0+z2)YM (v, z2)w = (z2+z0) YM (Y (u,z0)v,z2)w (see [Dong et al. 1998a, (3.5)]), where u ∈ V r , v ∈ V , w ∈ M, and k is a nonnegative k+r/T integer such that z YM (u, z)w ∈ M[[z]]. Let (M, YM ) and (N, YN ) be weak g-twisted V -modules. A homomorphism of M to N is a linear map f : M → N such that f YM (v, z) = YN (v, z) f for all v ∈ V . Let ގ be the set of nonnegative integers. ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 473

1 Deﬁnition 2.2. A T ގ-graded weak g-twisted V -module M is a weak g-twisted 1 L V -module with a ގ-grading M = ∈ 1 M(n) such that T n T ގ

(2-5) vm M(n) ⊂ M(n+wt(v)−m−1) for any homogeneous vectors v ∈ V .

1 A T ގ-graded weak g-twisted V -module here is called an admissible g-twisted 1 V -module in [Dong et al. 1998a]. Without loss we can shift the grading of a T ގ- graded weak g-twisted V -module M so that M(0) 6= 0 if M 6= 0. We call such an M(0) the top level of M. Definition 2.3. A g-twisted V -module M is a weak g-twisted V -module with a L ރ-grading M = λ∈ރ Mλ, where Mλ = {w ∈ M | L(0)w = λw}. Moreover, each Mλ is a finite dimensional space and for any fixed λ, Mλ+n/T = 0 for all sufficiently small integers n. A g-twisted V -module is sometimes called an ordinary g-twisted V -module. By 1 [Dong et al. 1998a, Lemma 3.4], any g-twisted V -module is a T ގ-graded weak g-twisted V -module. Indeed, assume that M is a g-twisted V -module. For each λ ∈ ރ with Mλ 6= 0, let λ0 = λ + m/T be such that m ∈ ޚ is minimal subject to 6= = L Mλ0 0. Let 3 be the set of all such λ0 and let M(n) λ∈3 Mn+λ. Then M(n) satisfies the condition in Definition 2.2. Thus we have the following inclusions. { } ⊂ { 1 } g-twisted V -modules T ގ-graded weak g-twisted V -modules ⊂ {weak g-twisted V -modules}

1 Deﬁnition 2.4. A vertex operator algebra V is said to be g-rational if every T ގ- graded weak g-twisted V -module is semisimple, that is, a direct sum of simple 1 T ގ-graded weak g-twisted V -modules. Let M be a weak g-twisted V -module. The next lemma is a twisted version of [Li 2001, Lemma 3.12]. In fact, using the associativity formula (2-4) we can prove it by essentially the same argument as in [Li 2001]. Lemma 2.5. Let u ∈ V r , v ∈ V s, w ∈ M, and k be a nonnegative integer such that k+r/T ∈ [[ ]] ∈ r + ∈ s + z YM (u, z)w M z . Let p T ޚ, q T ޚ, and N be a nonnegative N+1+q integer such that z YM (v, z)w ∈ M[[z]]. Then

(2-6) N ∞ X X p−k−r/T k+r/T u v w = (u − − − + v) + + + − w. p q i j p k r/T i j q k r/T i j i=0 j=0

Conversely, (u pv)q w can be written as a linear combination of some vectors of the form ui v j w. 474 KENICHIRO TANABE AND HIROMICHI YAMADA

∈ r ∈ s ∈ ∈ ∈ r+s + Lemma 2.6. Let u V , v V , w M. Then for p ޚ and q T ޚ, the ∈ r + ∈ s + vector (u pv)q w is a linear combination of ui v j w with i T ޚ and j T ޚ.

= { | ∈ r + ∈ s + } ∈ r + Proof. Let X span ui v j w i T ޚ, j T ޚ . We use (2-2). Take m T ޚ such that um+i w = 0 for i ≥ 0. Let N ∈ ޚ be such that u N+i v = 0 for i > 0. If p > N, then u pv = 0 and the assertion is trivial. Assume that p ≤ N. For j = 0, 1,..., N − p, let l = p + j and n = q − m − j in (2-2). Then

∞ ∞ X m X i p + j (u + + v) − − w = (−1) u + + − v − − + w. i p j i q j i i p m j i q m j i i=0 i=0

The right hand side of this equation is contained in X. Consider the left hand side for each of j = N − p, N − p − 1,..., 1, 0. Then we see that (u N v)q−N+pw ∈ X, (u N−1v)q−N+p+1w ∈ X, ... , and (u pv)q w ∈ X.

For subsets A, B of V and a subset X of M, set A · X = span{unw | u ∈ A, w ∈ ∈ 1 } · = { | ∈ ∈ ∈ } X, n T ޚ and A B span unv u A, v B, n ޚ . Then it follows from (2-6) that A · (B · X) ⊂ (A · B) · X (see also [Yamauchi 2004, (2.2)]). For a vector w ∈ M, this in particular implies that V · w is a weak g-twisted V -submodule of M. If w is an eigenvector for L(0), then V · w is a direct sum of eigenspaces for L(0). Each eigenspace is not necessarily of finite dimension. Thus V · w is not a g-twisted module in general. This subject was discussed in [Abe et al. 2004; Buhl 2002; Yamauchi 2004]. We will review it later in this section. Zhu [1996] introduced an associative algebra A(V ) called the Zhu algebra for a vertex operator algebra V , which plays a crucial role in the study of representations for V . Later, Dong, Li and Mason [Dong et al. 1998a] constructed an associative algebra Ag(V ) called the g-twisted Zhu algebra in order to generalize Zhu’s theory to g-twisted representations for V . The definition of Ag(V ) is similar to that of r A(V ). Let V , g, T , and V be as before. Roughly speaking, Ag(V ) = V/Og(V ) is a quotient space of V with a binary operation ∗g. It is in fact an associative r 0 algebra with respect to ∗g. If r 6= 0, then V ⊂ Og(V ). Thus Ag(V ) = (V + Og(V ))/Og(V ). For the case g = 1, see (5-1) in Section 5. A certain Lie algebra V [g] was considered in [Dong et al. 1998a]. Any weak g-twisted V -module is a module for the Lie algebra V [g] (see Lemma 5.1 of that reference). Moreover, for a V [g]-module M, the space (M) of lowest-weight vectors with respect to V [g] was defined. If M is a weak g-twisted V -modules, then (M) is the set of w ∈ M such that vwt(v)−1+nw = 0 for all homogeneous ∈ ∈ 1 7→ vectors v V and 0 < n T ޚ. The map v o(v) for homogeneous vectors 0 v ∈ V induces a representation of the associative algebra Ag(V ) on (M), where = 1 o(v) vwt(v)−1. If M is a T ގ-graded weak g-twisted V -module, then the top level ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 475

1 M(0) is contained in (M). In the case where M is a simple T ގ-graded weak g- twisted V -module, M(0) = (M) and M(0) is a simple Ag(V )-module (see [Dong et al. 1998a, Proposition 5.4]). 1 [ ] For any Ag(V )-module U, a certain T ގ-graded V g -module M(U) such that M(U)(0) = U was deﬁned (see [Dong et al. 1998a, (6.1)]). Let W be the subspace of M(U) spanned by the coefﬁcients of

wt(u)−1+δr +r/T (z0 + z2) YM (u, z0 + z2)YM (v, z2)w wt(u)−1+δr +r/T − (z2 + z0) YM (Y (u, z0)v, z2)w for all homogeneous u ∈ V r , v ∈ V , w ∈ U (see [Dong et al. 1998a, (6.3)]). Set M¯ (U) = M(U)/U(V [g])W, which is a quotient module of M(U) by the V [g]- submodule generated by W. The following results will be necessary in Sections 3 and 5.

¯ 1 Theorem 2.7 [Dong et al. 1998a, Theorem 6.2]. M(U) is a T ގ-graded weak g- ¯ twisted V -module such that its top level M(U)(0) is equal to U and such that it has the following universal property: for any weak g-twisted V -module M and any homomorphism ϕ : U → (M) of Ag(V )-modules, there is a unique homomorphism ϕ¯ : M¯ (U) → M of weak g-twisted V -modules which is an extension of ϕ.

1 [ ] Let J be the sum of all T ގ-graded V g -submodules of M(U) which intersect = 1 [ ] trivially with U. Since M(U)(0) U, it is a unique T ގ-graded V g -submodule of M(U) being maximal subject to J ∩U =0. The principal point is that U(V [g])W ⊂ J. Set L(U) = M(U)/J.

Theorem 2.8 [Dong et al. 1998a, Theorem 6.3]. L(U) is a 1 ގ-graded weak g- ∼ T twisted V -module such that (L(U)) = U as Ag(V )-modules.

1 : → Remark 2.9. If M is a T ގ-graded weak g-twisted V -module and ϕ U M(0) is ¯ a homomorphism of Ag(V )-modules, then the homomorphism ϕ¯ : M(U) → M 1 of weak g-twisted V -modules in Theorem 2.7 preserves the T ގ-grading. In- ¯ = { | ∈ ∈ 1 } ¯ deed, M(U) span vnU v V, n T ޚ by (2-6), since M(U) is generated 1 ¯ ⊂ by U as a T ގ-graded weak g-twisted V -module. By (2-5), vwt(v)−1−n M(U)(0) ¯ ∈ ∈ 1 ¯ = M(U)(n) for any homogeneous v V and n T ޚ. Since M(U)(0) U, it fol- ¯ lows that M(U)(n) is spanned by vwt(v)−1−nU for all homogeneous v ∈ V . Now, ϕ(v¯ wt(v)−1−nU) = vwt(v)−1−nϕ(¯ U) is contained in vwt(v)−1−n M(0) ⊂ M(n). Hence ¯ ¯ ϕ(¯ M(U)(n)) ⊂ M(n) as required. In the case where both of M(U) and M are ordinary g-twisted V -modules, ϕ¯ becomes a homomorphism of ordinary g-twisted V -modules since ϕ¯ commutes with L(0). 476 KENICHIRO TANABE AND HIROMICHI YAMADA

1 Lemma 2.10. Let U be an Ag(V )-module. Let S be a T ގ-graded weak g-twisted V -module such that it is generated by its top level S(0) and such that S(0) is isomorphic to U as an Ag(V )-module. Then there is a surjective homomorphism → 1 S L(U) of weak g-twisted V -modules which preserves the T ގ-grading.

Proof. By Theorem 2.7 and Remark 2.9, an isomorphism ϕ : U → S(0) of Ag(U)- modules can be extended to a surjective homomorphism ϕ¯ : M¯ (U) → S of weak 1 ¯ ¯ g-twisted V -modules which preserves the T ގ-grading. The kernel Ker ϕ of ϕ ¯ L ¯ intersects trivially with M(U)(0) and so is contained in 0

(1) Ag(V ) is a finite dimensional semisimple associative algebra. 1 (2) V has only finitely many isomorphism classes of simple T ގ-graded weak g- twisted V -modules. 1 (3) Every simple T ގ-graded weak g-twisted V -module is an ordinary g-twisted V -modules. In case of g = 1, the above argument reduces to the untwisted case. In particular, Ag(V ) is identical with the original Zhu algebra A(V ) if g = 1. There is an important intrinsic property of a vertex operator algebra, namely, the C2-cofiniteness. Let C2(V ) = span{u−2v | u, v ∈ V }. More generally, we set C2(M) = span{u−2w | u ∈ V, w ∈ M} for a weak V -module M. If the dimension of the quotient space V/C2(V ) is finite, V is said to be C2-cofinite. Similarly, a weak V -module M is said to be C2-cofinite if M/C2(M) is of finite dimension. The notion of C2-cofiniteness of a vertex operator algebra was first introduced by Zhu [1996]. The subspace C2(M) of a weak V -module M was studied in [Li 1999b]. We refer the reader to [Nagatomo and Tsuchiya 2005] also.

Theorem 2.13 [Dong et al. 2000, Proposition 3.6]. If V is C2-coﬁnite, then Ag(V ) is of ﬁnite dimension. L∞ If V = n=0 Vn and V0 = ރ1, then V is said to be of CFT type. Here Vn denotes the homogeneous subspace of weight n, that is, the eigenspace of L(0) = ω1 with eigenvalue n. ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 477

Theorem 2.14 [Yamauchi 2004, Lemma 3.3]. Suppose V is C2-coﬁnite and of CFT type. Choose a ﬁnite dimensional L(0)-invariant and g-invariant subspace U of V such that V = U + C2(V ). Let W be a weak g-twisted V -module generated by a vector w. Then W is spanned by the vectors of the form u1 u2 ··· uk w −n1 −n2 −nk ··· − i ∈ = ∈ 1 with n1 > n2 > > nk > N and u U, i 1, 2,..., k, where N T ޚ is a constant such that umw = 0 for all u ∈ U and m ≥ N.

Theorem 2.15 [Yamauchi 2004, Corollaries 3.8 and 3.9]. Suppose V is C2-coﬁnite and of CFT type. Then the following assertions hold. 1 (1) Every weak g-twisted V -module is a T ގ-graded weak g-twisted V -module. (2) Every simple weak g-twisted V -module is a simple ordinary g-twisted V - module.

Remark 2.16. Suppose V is C2-cofinite and of CFT type. Let M be a weak g- twisted V -module and w1, ..., wk be eigenvectors of L(0) in M. Then the weak g-twisted V -submodule W generated by w1, ..., wk is an ordinary g-twisted V - module. Indeed, W is a direct sum of eigenspaces for L(0) and each homogeneous subspace is of finite dimension by Theorem 2.14. For the untwisted case, that is, the case g = 1, we refer the reader to [Abe et al. 2004; Buhl 2002; Dong et al. 1997; Li 1999b]. A spanning set for a vertex operator algebra was first studied in [Gaberdiel and Neitzke 2003, Proposition 8].

3. The fixed point subalgebra (V√ )τ 2A2 In this section we fix notation. We tend to follow the notation in [Dong et al. 2004; Kitazume et al. 2000a; Kitazume et al. 2003] unless otherwise specified. We also recall certain properties of the lattice vertex operator algebra V√ associated with √ 2A2 2 times an ordinary root lattice of type A2 and its subalgebras (see [Dong et al. 2004; Kitazume et al. 2000a; Kitazume et al. 2003; Kitazume et al. 2000b]). Let α1, α2 be the simple roots of type √A2 and set α0 =−(α1+α2). Thus hαi , αi i= 2 and hαi , α j i = −1 if i 6= j. Set βi = 2αi and let L = ޚβ1 + ޚβ2 be the lattice ⊥ spanned by β1 and β2. We denote the cosets of L in its dual lattice L = {α ∈ ޑ ⊗ޚ L | hα, Li ⊂ ޚ} as follows. −β + β β − β L0 = L, L1 = 1 2 + L, L2 = 1 2 + L, 3 3

β2 β0 β1 L = L, La = + L, Lb = + L, Lc = + L, 0 2 2 2 (i, j) j L = Li + L ∼ for i = 0, a, b, c and j = 0, 1, 2, where {0, a, b, c} = ޚ2 ×ޚ2 is Klein’s four-group. Note that L(i, j), i ∈ {0, a, b, c}, j ∈ {0, 1, 2} are all the cosets of L in L⊥ and ⊥ ∼ L /L = ޚ2 × ޚ2 × ޚ3. 478 KENICHIRO TANABE AND HIROMICHI YAMADA

We adopt the standard notation for the vertex operator algebra (VL , Y ( · , z)) associated with the lattice L (see [Frenkel et al. 1988]). In particular, h = ރ ⊗ޚ L is an abelian Lie algebra, hˆ = h ⊗ ރ[t, t−1] ⊕ ރc is the corresponding afﬁne Lie algebra, M(1) = ރ[α(n) ; α ∈ h, n < 0], where α(n) = α ⊗ tn, is the unique simple hˆ-module such that α(n)1 = 0 for all α ∈ h and n > 0 and c = 1. As a vector space VL = M(1) ⊗ ރ[L] and for each v ∈ VL , a vertex operator Y (v, z) = P −n−1 −1 n∈ޚ vn z ∈ End(VL )[[z, z ]] is deﬁned. The vector 1 = 1 ⊗ 1 is called the vacuum vector. In our case hα, βi ∈ 2ޚ for any α, β ∈ L. Thus the twisted group algebra ރ{L} of [Frenkel et al. 1988] is naturally isomorphic to the ordinary group algebra ރ[L]. There are exactly 12 inequivalent simple VL -modules, which are represented by VL(i, j) , i = 0, a, b, c and j = 0, 1, 2 (see [Dong 1993]). We use the symbol eα, α ∈ L⊥ to denote a basis of ރ{L⊥}. We consider the following three isometries of (L, h·, ·i).

τ : β1 → β2 → β0 → β1,

(3-1) σ : β1 → β2, β2 → β1,

θ : βi → −βi , i = 1, 2. Note that τ is fixed-point-free and of order 3. The isometries τ, σ, and θ of L can be extended linearly to isometries of L⊥. Moreover, the isometry τ lifts naturally to an automorphism of VL : 1 k β 1 k τβ α (−n1) ··· α (−nk)e 7−→ (τα )(−n1) ··· (τα )(−nk)e . By abuse of notation, we denote it by τ also. We can consider the action of τ on VL(i, j) in a similar way. We apply the same argument to σ and θ. Our purpose τ is the classification of simple modules for the fixed point subalgebra VL = {v ∈ VL | τv = v} of VL by the automorphism τ. For a simple VL -module (U, YU ), let (U ◦ τ, YU◦τ ) be a new VL -module such that U ◦τ = U as vector spaces and YU◦τ (v, z) = YU (τv, z) for v ∈ VL [Dong et al. 2000]. Then U 7→ U ◦ τ induces a permutation on the set of simple VL -modules. If U and U ◦ τ are equivalent VL -modules, U is said to be τ-stable. The following lemma is a straightforward consequence of the definition of VL(i, j) .

Lemma 3.1. (1) VL(0, j) , j = 0, 1, 2 are τ-stable. (2) VL(a, j) ◦ τ = VL(c, j) , VL(c, j) ◦ τ = VL(b, j) , and VL(b, j) ◦ τ = VL(a, j) , j = 0, 1, 2. A family of simple twisted modules for lattice vertex operator algebras was constructed in [Dong and Lepowsky 1996; Lepowsky 1985]. Following Dong and j τ Lepowsky, three inequivalent simple τ-twisted VL -modules (VL (τ), Y ( · , z)), j = 0, 1, 2 were studied in [Dong et al. 2004; Kitazume et al. 2003]. By the preceding j τ lemma and [Dong et al. 2000, Theorem 10.2], we know that (VL (τ), Y ( · , z)), ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 479 j = 0, 1, 2, are all the inequivalent simple τ-twisted VL -modules. Similarly, there 2 j 2 τ 2 are exactly three inequivalent simple τ -twisted VL -modules (VL (τ ), Y ( · , z)), j = 0, 1, 2. j τ j 2 τ 2 We use the same notation for (VL (τ), Y ( · , z)) and (VL (τ ), Y ( · , z)) as in [Dong et al. 2004, Section 4]. Thus

j = [ ] ⊗ VL (τ) S τ Tχ j , = where Tχ j , j 0, 1, 2, is the one-dimensional representation of a certain central extension of L affording the character χ j . Let

= 1 + 2 + = 1 + + 2 h1 3 (β1 ξ β2 ξβ0), h2 3 (β1 ξβ2 ξ β0).

i i−1 Then τhi =ξ hi , hh1, h1i=hh2, h2i=0, and hh1, h2i=2. Moreover, βi =ξ h1+ 2(i−1) ξ h2, i = 0, 1, 2. As a vector space, S[τ] is isomorphic to a polynomial algebra with variables h1(1/3 + n), h2(2/3 + n), n ∈ ޚ<0. The isometry τ acts on S[τ] by = j τh j ξ h j . We define the action of τ on Tχ j to be the identity. The weight in S[τ] is given by wt hi (i/3 + n) = −i/3 − n, i = 1, 2 and wt 1 = 1/9. The weight j of any element of Tχ j is defined to be 0. Note that the weight in VL (τ) is identical with the eigenvalue for the action of the coefficient of z−2 in the τ-twisted vertex τ operator Y (ω, z), where ω denotes the Virasoro element of VL . 2 j 2 τ 2 The simple τ -twisted VL -modules (VL (τ ), Y ( · , z)), j = 0, 1, 2 are

j V (τ 2) = S[τ 2] ⊗ T 0 , L χ j where Tχ0 , j = 0, 1, 2, are the one-dimensional representations of a certain central j 0 2 extension of L affording the character χ j . Moreover, S[τ ] is isomorphic to a 0 + 0 + ∈ polynomial algebra with variables h1(1/3 n), h2(2/3 n), n ޚ<0 as a vector 0 = 0 = 2 0 = i 0 = space, where h1 h2 and h2 h1. Thus τ hi ξ hi , i 1, 2. The action of τ 2 0 2i 0 on S[τ ] is given by τhi = ξ hi , i = 1, 2. The action of τ on Tχ0 is defined to be 2 0 j the identity. The weight in S[τ ] is given by wt hi (i/3 + n) = −i/3 − n, i = 1, 2 and wt 1 = 1/9. The weight of any element of T 0 is defined to be 0. The weight χ j j 2 −2 in VL (τ ) is identical with the eigenvalue for the action of the coefficient of z in the τ 2-twisted vertex operator Y τ 2 (ω, z). By Lemma 3.1,[Dong and Mason 1997, Theorem 4.4], and [Dong and Yam- skulna 2002, Theorem 6.14],

ε VL(0, j) (ε) = {v ∈ VL(0, j) | τv = ξ v}, j, ε = 0, 1, 2

τ are inequivalent simple VL -modules. For each of j = 0, 1, 2, we have that VL(i, j) , τ i = a, b, c are equivalent simple VL -modules. Moreover, VL(c, j) , j = 0, 1, 2 are τ inequivalent simple VL -modules. From [Miyamoto and Tanabe 2004, Theorem 2], 480 KENICHIRO TANABE AND HIROMICHI YAMADA it follows that

j j ε VL (τ)(ε) = {v ∈ VL (τ) | τv = ξ v}, j, ε = 0, 1, 2 τ 2 are inequivalent simple VL -modules. Similar assertions hold for simple τ -twisted modules, namely,

j 2 j 2 2 ε VL (τ )(ε) = {v ∈ VL (τ ) | τ v = ξ v}, j, ε = 0, 1, 2 τ τ are inequivalent simple VL -modules. In this way we obtain 30 simple VL -modules. τ These 30 simple VL -modules are inequivalent by [Miyamoto and Tanabe 2004, Theorem 2]. We summarize the result as follows.

τ Lemma 3.2. The following 30 simple VL -modules are inequivalent.

(1) VL(0, j) (ε), j, ε = 0, 1, 2,

(2) VL(c, j) , j = 0, 1, 2, j (3) VL (τ)(ε), j, ε = 0, 1, 2, j 2 (4) VL (τ )(ε), j, ε = 0, 1, 2. τ We consider the structure of VL in detail. Set √ √ √ √ = 2α + − 2α = 2α − − 2α = 1 − 2 − x(α) e e , y(α) e e , w(α) 2 α( 1) x(α) for α ∈ {±α0, ±α1, ±α2} and let = 1 − 2 + − 2 + − 2 ω 6 α1( 1) α2( 1) α0( 1) , ˜ 1 = 1 + + ˜ 2 = − ˜ 1 ω 5 w(α1) w(α2) w(α0) , ω ω ω , 1 = 1 2 = ˜ 1 − 1 ω 4 w(α1), ω ω ω .

1 2 Then ω is the Virasoro element of VL and ω˜ , ω˜ are mutually orthogonal conformal vectors of central charge 6/5, 4/5 respectively. The subalgebra Vir(ω˜ i ) generated by ω˜ i is isomorphic to the Virasoro vertex operator algebra of given central charge, namely, Vir(ω˜ 1) =∼ L(6/5, 0) and Vir(ω˜ 2) =∼ L(4/5, 0). Moreover, ω˜ 1 is a sum of two conformal vectors ω1 and ω2 of central charge 1/2 and 7/10 respectively and ω1, ω2 and ω˜ 2 are mutually orthogonal. Note that ω˜ 2 was denoted by ω3 in [Dong et al. 2004; Kitazume et al. 2000a; Kitazume et al. 2003; Kitazume et al. 2000b]. Such a decomposition of the Virasoro element of a lattice vertex operator algebra into a sum of mutually orthogonal conformal vectors was ﬁrst studied in [Dong et al. 1998b]. Set i = { ∈ | ˜ 2 = } i = { ∈ | ˜ 2 = 2 } = Mk v VLi (ω )1v 0 , Wk v VLi (ω )1v 5 v , i 0, a, b, c, ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 481

j 1 2 Mt = {v ∈ VL j | (ω )1v = (ω )1v = 0}, j = { ∈ | 1 = 2 = 3 } = Wt v VL j (ω )1v 0, (ω )1v 5 v , j 0, 1, 2.

0 0 i i Then Mk and Mt are simple vertex operator algebras. Moreover, {Mk, Wk ; i = j j 0, a, b, c} and {Mt , Wt ; j = 0, 1, 2} are complete sets of representatives of iso- 0 0 morphism classes of simple modules for Mk and Mt , respectively (see [Kitazume et al. 2000a; Kitazume et al. 2000b; Lam and Yamada 2000]). As Vir(ω1) ⊗ Vir(ω2)-modules,

0 =∼ 1 ⊗ 7 ⊕ 1 1 ⊗ 7 3 Mk L( 2 , 0) L( 10 , 0) L( 2 , 2 ) L 10 , 2 ) , a =∼ b =∼ 1 1 ⊗ 7 7 Mk Mk L( 2 , 16 ) L( 10 , 16 ), c ∼ 1 1 7 1 7 3 Mk = L( , ) ⊗ L( , 0) ⊕ L( , 0) ⊗ L( , ) , (3-2) 2 2 10 2 10 2 0 =∼ 1 ⊗ 7 3 ⊕ 1 1 ⊗ 7 1 Wk L( 2 , 0) L( 10 , 5 ) L( 2 , 2 ) L( 10 , 10 ) , a =∼ b =∼ 1 1 ⊗ 7 3 Wk Wk L( 2 , 16 ) L( 10 , 80 ), c =∼ 1 1 ⊗ 7 3 ⊕ 1 ⊗ 7 1 Wk L( 2 , 2 ) L( 10 , 5 ) L( 2 , 0) L( 10 , 10 ) , and as Vir(ω˜ 2)-modules,

0 ∼ 4 4 1 ∼ 2 ∼ 4 2 Mt = L( , 0) ⊕ L( , 3), Mt = Mt = L( , ), (3-3) 5 5 5 3 0 =∼ 4 2 ⊕ 4 7 1 =∼ 2 =∼ 4 1 Wt L( 5 , 5 ) L( 5 , 5 ), Wt Wt L( 5 , 15 ). Furthermore,

∼ i j i j (3-4) VL(i, j) = (Mk ⊗ Mt ) ⊕ (Wk ⊗ Wt )

0 0 as Mk ⊗ Mt -modules. In particular, ∼ 0 0 0 0 (3-5) VL = (Mk ⊗ Mt ) ⊕ (Wk ⊗ Wt ).

j 1 0 0 j Note that Mt = {v ∈ VL j | (ω˜ )1v = 0} and that Mk , Wk and Mt , j = 0, 1, 2 are j τ-invariant. However, Wt , j = 0, 1, 2 are not τ-invariant. 0 0 The fusion rules for Mk and Mt were determined in [Lam and Yamada 2000] and [Miyamoto 2001], respectively. They are

i j i+ j i j i+ j i j i+ j i+ j (3-6) Mk × Mk = Mk , Mk × Wk = Wk , Wk × Wk = Mk + Wk for i, j = 0, a, b, c and

i j i+ j i j i+ j i j i+ j i+ j (3-7) Mt × Mt = Mt , Mt × Wt = Wt , Wt × Wt = Mt + Wt for i, j = 0, 1, 2. 482 KENICHIRO TANABE AND HIROMICHI YAMADA

The following two weight-three vectors are important.

J = w(α1)0w(α2) − w(α2)0w(α1) 1 = − β (−2)(β − β )(−1) + β (−2)(β − β )(−1) + β (−2)(β − β )(−1) 6 1 2 0 2 0 1 0 1 2 − (β2 − β0)(−1)y(α1) − (β0 − β1)(−1)y(α2) − (β1 − β2)(−1)y(α0), 1 K = − (β − β )(−1)(β − β )(−1)(β − β )(−1) 9 1 2 2 0 0 1 + (β2 − β0)(−1)x(α1) + (β0 − β1)(−1)x(α2) + (β1 − β2)(−1)x(α0).

0 τ 0 Let M(0) = (Mk ) = {u ∈ Mk | τu = u}. The vertex operator algebra M(0) was studied in [Dong et al. 2004]. Among other things, the classiﬁcation of simple modules, the rationality and the C2-coﬁniteness for M(0) were established. It is known that M(0) is a W3 algebra of central charge 6/5 with the Virasoro element ω˜ 1. In fact, M(0) is generated by ω˜ 1 and J. The following equations hold [Dong et al. 2004, (3.1)].

J5 J = −84 · 1,

J4 J = 0, 1 J3 J = −420ω˜ , (3-8) 1 1 J2 J = −210(ω˜ )0ω˜ , 1 1 1 1 1 J1 J = 9(ω˜ )0(ω˜ )0ω˜ − 240(ω˜ )−1ω˜ , 1 1 1 1 1 1 1 J0 J = 22(ω˜ )0(ω˜ )0(ω˜ )0ω˜ − 120(ω˜ )0(ω˜ )−1ω˜ .

1 1 Let L (n) = (ω˜ )n+1 and J(n) = Jn+2 for n ∈ ޚ, so that the weight of these operators is wt L1(n) = wt J(n) = −n. Then

3 1 1 1 m − m 6 (3-9) [L (m), L (n)] = (m − n)L (m + n) + · · δm+n , 12 5 ,0 (3-10) [L1(m), J(n)] = (2m − n)J(m + n),

(3-11) [J(m), J(n)] = (m−n)22(m+n+2)(m+n+3)+35(m+2)(n+2)L1(m+n) X X −120(m−n) L1(k)L1(m+n−k)+ L1(m+n−k)L1(k) k≤−2 k≥−1 − 7 2− 2− 10 m(m 1)(m 4)δm+n,0. 0 The vertex operator algebra Mt is known as a 3-State Potts model. It is a W3 algebra of central charge 4/5 with the Virasoro element ω˜ 2 and is generated by ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 483

2 2 0 ω˜ and K . Both of ω˜ and K are ﬁxed by τ, so that τ is the identity on Mt . The 0 rationality of Mt was established in [Kitazume et al. 2000b] and the C2-coﬁniteness 0 of Mt follows from [Buhl 2002]. By a direct calculation, we can verify that

K5 K = 104 · 1,

K4 K = 0, 2 K3 K = 780ω˜ , (3-12) 2 2 K2 K = 390(ω˜ )0ω˜ , 2 2 2 2 2 K1 K = −27(ω˜ )0(ω˜ )0ω˜ + 480(ω˜ )−1ω˜ , 2 2 2 2 2 2 2 K0 K = −46(ω˜ )0(ω˜ )0(ω˜ )0ω˜ + 240(ω˜ )0(ω˜ )−1ω˜ .

2 2 Let L (n) = (ω˜ )n+1 and K (n) = Kn+2 for n ∈ ޚ. Then

3 2 2 2 m − m 4 (3-13) [L (m), L (n)] = (m − n)L (m + n) + · · δm+n , 12 5 ,0

(3-14) [L2(m), K (n)] = (2m − n)K (m + n),

(3-15) [K (m), K (n)] = −(m−n)46(m+n+2)(m+n+3) + 65(m+2)(n+2)L2(m+n) X X +240(m−n) L2(k)L2(m+n−k) + L2(m+n−k)L2(k) k≤−2 k≥−1 + 13 2− 2− 15 m(m 1)(m 4)δm+n,0. √ 1 Remark 3.3. Let Ln = L (n), Wn = −1/210J(n), and c = 6/5. Then the commutation relations above coincide with (2.1) and (2.2) of [Bouwknegt et al. 2 1996].√ The same commutation relations also hold if we set Ln = L (n), Wn = K (n)/ 390, and c = 4/5. Let us review the 20 inequivalent simple M(0)-modules studied in [Dong et al. 0 2004]. Among those simple M(0)-modules, eight of them appear in simple Mk - modules, namely,

0 ε 0 ε M(ε) = {u ∈ Mk | τu = ξ u}, W(ε) = {u ∈ Wk | τu = ξ u}

c c for ε = 0, 1, 2, Mk and Wk . The remaining 12 simple M(0)-modules appear in 2 simple τ-twisted or τ -twisted VL -modules. Let

0 2 ε MT (τ)(ε) = {u ∈ VL (τ) | (ω˜ )1u = 0, τu = ξ u}, = { ∈ 0 | ˜ 2 = 2 = ε } WT (τ)(ε) u VL (τ) (ω )1u 5 u, τu ξ u . 484 KENICHIRO TANABE AND HIROMICHI YAMADA

Then MT (τ)(ε), WT (τ)(ε), ε = 0, 1, 2 are inequivalent simple M(0)-modules. Similarly,

2 0 2 2 2 ε MT (τ )(ε) = {u ∈ VL (τ ) | (ω˜ )1u = 0, τ u = ξ u}, 2 = { ∈ 0 2 | ˜ 2 = 2 2 = ε } WT (τ )(ε) u VL (τ ) (ω )1u 5 u, τ u ξ u for ε = 0, 1, 2 are inequivalent simple M(0)-modules. In [Dong et al. 2004], it c c 2 was shown that M(ε), W(ε), Mk , Wk , MT (τ)(ε), WT (τ)(ε), MT (τ )(ε), and 2 WT (τ )(ε), ε = 0, 1, 2 form a complete set of representatives of isomorphism classes of simple M(0)-modules. τ Let us describe the structure of the ﬁxed point subalgebra VL . By the deﬁnition 0 τ 0 0 of M(0) and Mt , we see that VL ⊃ M(0) ⊗ Mt . Since both of M(0) and Mt are 0 ε rational, M(0)⊗ Mt is also rational. Thus VL (ε) = {u ∈ VL | τu = ξ u}, ε = 0, 1, 2 0 can be decomposed into a direct sum of simple modules for M(0) ⊗ Mt . Any 0 simple module for M(0) ⊗ Mt is of the form A ⊗ B, where A and B are simple 0 ∼ 0 modules for M(0) and Mt , respectively. By (3-5), it follows that B = Mt or 0 Wt . Moreover, VL (ε) contains the simple M(0)-modules M(ε) and W(ε). The 1 eigenvalues of (ω˜ )1 in M(ε) (resp. W(ε)) are integers (resp. of the form 3/5 + n, 2 0 0 n ∈ ޚ), while the eigenvalues of (ω˜ )1 in Mt (resp. Wt ) are integers (resp. of the 1 2 form 2/5 + n, n ∈ ޚ). Since the eigenvalues of ω1 = (ω˜ )1 + (ω˜ )1 in VL are integers, we conclude that

∼ 0 0 (3-16) VL (ε) = (M(ε) ⊗ Mt ) ⊕ (W(ε) ⊗ Wt )

0 as M(0) ⊗ Mt -modules, ε = 0, 1, 2. In particular, τ ∼ 0 0 (3-17) VL = (M(0) ⊗ Mt ) ⊕ (W(0) ⊗ Wt ).

0 0 0 0 τ From now on we set M = M(0) ⊗ Mt and W = W(0) ⊗ Wt . Thus VL = ∼ 0 0 VL (0) = M ⊕ W . Let

P = y(α1) + y(α2) + y(α0).

1 2 1 Then we can verify that (ω˜ )n P = (ω˜ )n P = 0 for n ≥ 2, (ω˜ )1 P = (8/5)P, and 2 0 0 (ω˜ )1 P = (2/5)P. Moreover, Jn P = Kn P = 0 for n ≥ 2. Thus W is a simple M - module with P a highest-weight vector of weight (8/5, 2/5). The vertex operator τ 1 2 algebra VL is generated by ω˜ , ω˜ , J, K and P. τ Theorem 3.4. VL is a simple C2-cofinite vertex operator algebra. 0 0 Proof. We know that M(0) and Mt are C2-cofinite. Thus M is also C2-cofinite. Since W 0 is generated by P as an M0-module, it follows from [Buhl 2002] that τ τ VL is C2-cofinite. By [Dong and Mason 1997, Theorem 4.4], VL is simple. ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 485

Following the outline of the argument in [Dong et al. 2004; Kitazume et al. j 2003], we discuss the structure of the simple τ-twisted VL -modules VL (τ), j = 0 0 0, 1, 2 as τ-twisted Mk ⊗ Mt -modules. Furthermore, we correct an error in [Dong j et al. 2004; Kitazume et al. 2003] concerning a decomposition of VL (τ) for j = 0 6= ∈ [ ] 1, 2. We ﬁrst consider VL (τ). Let 0 v Tχ0 and 1 be the identity of S τ . Then ⊗ ∈ [ ] ⊗ = 0 0 ⊂ τ 0 1 v S τ Tχ0 VL (τ). Since Mt VL , we can decompose VL (τ) into a 0 direct sum of simple Mt -modules. By a direct calculation, we can verify that ˜ 2 ⊗ = ˜ 2 − 1 ⊗ = 2 − 1 ⊗ (ω )1(1 v) 0,(ω )1(h2( 3 ) v) 5 h2( 3 ) v. 0 0 0 Thus we see that Mt and Wt appear as direct summands. Since VL (τ) is simple 0 0 0 0 as a τ-twisted VL -module, (3-5) and the fusion rule Wt × Wt = Mt + Wt (see 0 0 0 0 (3-7)) imply that any simple Mt -submodule of VL (τ) is isomorphic to Mt or Wt . Hence 0 ∼ 0 0 0 0 (3-18) VL (τ) = (MT (τ) ⊗ Mt ) ⊕ (WT (τ) ⊗ Wt ) 0 0 as τ-twisted Mk ⊗ Mt -modules, where 0 0 2 MT (τ) = {u ∈ VL (τ) | (ω˜ )1u = 0}, 0 = { ∈ 0 | ˜ 2 = 2 } WT (τ) u VL (τ) (ω )1u 5 u . 0 0 0 The τ-twisted Mk -modules MT (τ) and WT (τ) are simple. Indeed, if N is a τ- 0 0 0 0 0 twisted Mk -submodule of MT (τ), then N ⊗Mt is a τ-twisted Mk ⊗Mt -submodule 0 0 0 0 of MT (τ) ⊗ Mt . By (2-6), VL · (N ⊗ Mt ) = span{an(N ⊗ Mt ) | a ∈ VL , n ∈ ޑ} 0 0 0 0 is a τ-twisted VL -submodule of VL (τ). The fusion rule Wt × Mt = Wt and (3-5) 0 0 0 0 0 imply that VL ·(N ⊗ Mt ) is contained in (N ⊗ Mt )⊕(WT (τ)⊗ Wt ). Since VL (τ) 0 is a simple τ-twisted VL -module, we conclude that MT (τ) is a simple τ-twisted 0 Mk -module. 0 0 0 0 Because of the fusion rule Wt × Wt = Mt + Wt , we can not apply a similar 0 argument to WT (τ). Note that there are at most two inequivalent simple τ-twisted 0 Mk -modules by [Dong et al. 2004, Lemma 4.1] and [Dong et al. 2000, Theorem 0 0 1 10.2]. Note also that a weight in MT (τ) or in WT (τ) means an eigenvalue of (ω˜ )1. 0 0 First several terms of the characters of MT (τ) and WT (τ) can be calculated easily from (3-18) (see [Dong et al. 2004]).

0 1/9 1/9+2/3 1/9+1 1/9+4/3 ch MT (τ) = q + q + q + q + · · · , 0 2/45 2/45+1/3 2/45+2/3 2/45+1 ch WT (τ) = q + q + q + q + · · · . 0 0 Suppose WT (τ) is not a simple τ-twisted Mk -module. Let N be the τ-twisted 0 0 0 Mk -submodule of WT (τ) generated by the top level of WT (τ). Then the top level 0 of N is a one dimensional space of weight 2/45. If N is not a simple τ-twisted Mk - 0 module, then the sum U of all proper τ-twisted Mk -submodules of N is a unique 486 KENICHIRO TANABE AND HIROMICHI YAMADA

0 maximal τ-twisted Mk -submodule of N. The quotient N/U is a simple τ-twisted 0 Mk -module whose top level is of weight 2/45. Denote the top level of U by Uλ, where the weight λ is 2/45 + n/3 for some 1 ≤ n ∈ ޚ. Consider the τ-twisted Zhu 0 0 0 algebra Aτ (Mk ) of Mk . Since Uλ is a finite dimensional Aτ (Mk )-module, we can 0 choose a simple Aτ (Mk )-submodule S of Uλ. By [Dong et al. 1998a, Proposition 1 0 5.4 and Theorem 7.2], there is a simple 3 ގ-graded weak τ-twisted Mk -module R 0 with top level Rλ being isomorphic to S as an Aτ (Mk )-module. It follows from 0 [Yamauchi 2004, Corollary 3.8] that R is in fact a simple τ-twisted Mk -module. 0 Here we note that Mk is C2-cofinite and of CFT type by its structure (3-2). Since 0 the top levels of MT (τ), N/U, and R have different weight, they are inequivalent 0 0 simple τ-twisted Mk -modules. If N is a simple τ-twisted Mk -module, then it is 0 0 0 not equal to WT (τ) by our assumption. The quotient WT (τ)/N is a τ-twisted Mk - module and the weight of its top level, say µ is 2/45 + m/3 for some 1 ≤ m ∈ ޚ. 0 By a similar argument as above, we see that there is a simple τ-twisted Mk -module whose top level is of weight µ. Hence we have three inequivalent simple τ-twisted 0 Mk -modules in both cases. This contradicts the fact that there are at most two 0 0 inequivalent simple τ-twisted Mk -modules. Thus WT (τ) is a simple τ-twisted 0 Mk -module. 6= ∈ = j Next, let 0 v Tχ j , j 1, 2. From the definition of VL (τ) in [Dong et al. 2004; Kitazume et al. 2003], we can calculate that ˜ 2 ⊗ = 1 ⊗ ˜ 2 j = 2 j (ω )1(1 v) 15 (1 v), (ω )1u 3 u , √ j = − 2 ⊗ − − j − − 1 2 ⊗ 1 2 1 2 where u h1( 3 ) v ( 1) 3h2( 3 ) v. Thus Mt or Mt and Wt or Wt 0 j 1 2 1 appear as Mt -submodules of VL (τ). In order to distinguish Mt and Mt (resp. Wt 2 and Wt ), we need to know the action of K2 on these vectors (see [Kitazume et al. 2000b]). By a direct calculation, we can verify that ⊗ = − − j 2 ⊗ j = − j 52 j K2(1 v) ( 1) 9 (1 v), K2u ( 1) 9 u . 3− j 3− j j Hence Mt and Wt appear in VL (τ) for j = 1, 2. Let j = { ∈ j | ˜ 2 = 2 } MT (τ) u VL (τ) (ω )1u 3 u , j = { ∈ j | ˜ 2 = 1 } = WT (τ) u VL (τ) (ω )1u 15 u , j 1, 2.

j ∼ j 3− j j 3− j 0 0 Then, VL (τ) = (MT (τ) ⊗ Mt ) ⊕ (WT (τ) ⊗ Wt ) as τ-twisted Mk ⊗ Mt - j j modules for j = 1, 2. Moreover, MT (τ) and WT (τ), j = 1, 2 are simple τ-twisted 0 Mk -modules. 0 Recall that there are at most two inequivalent simple τ-twisted Mk -modules. j j j Looking at the smallest weight of MT (τ) and WT (τ), we see that the MT (τ), j 0 j = 0, 1, 2 are equivalent, and the WT (τ), j = 0, 1, 2 are equivalent, but MT (τ) and ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 487

0 0 0 WT (τ) are not equivalent. For simplicity, set MT (τ)= MT (τ) and WT (τ)= WT (τ). Then 0 ∼ 0 0 VL (τ) = (MT (τ) ⊗ Mt ) ⊕ (WT (τ) ⊗ Wt ), (3-19) j ∼ 3− j 3− j VL (τ) = (MT (τ) ⊗ Mt ) ⊕ (WT (τ) ⊗ Wt ), j = 1, 2,

0 0 as τ-twisted Mk ⊗ Mt -modules. 2 j 2 2 The structure of the simple τ -twisted VL -module VL (τ ), j = 0, 1, 2 as a τ - j twisted M0 ⊗ M0-module is similar to that of the case for V (τ). Let 0 6= v ∈ T 0 k t L χ0 and let 1 be the identity of S[τ 2]. Then

˜ 2 ⊗ = ˜ 2 0 − 1 ⊗ = 2 0 − 1 ⊗ (ω )1(1 v) 0,(ω )1(h2( 3 ) v) 5 h2( 3 ) v and so 0 2 ∼ 0 2 0 0 2 0 VL (τ ) = (MT (τ ) ⊗ Mt ) ⊕ (WT (τ ) ⊗ Wt ) 2 0 0 as τ -twisted Mk ⊗ Mt -modules, where

0 2 0 2 2 MT (τ ) = {u ∈ VL (τ ) | (ω˜ )1u = 0}, 0 2 = { ∈ 0 2 | ˜ 2 = 2 } WT (τ ) u VL (τ ) (ω )1u 5 u .

0 2 By a similar argument as in the τ-twisted case, we can show that MT (τ ) and 0 2 2 0 WT (τ ) are inequivalent simple τ -twisted Mk -modules. Take a nonzero v in T 0 , j = 1, 2. Then χ j ˜ 2 ⊗ = 1 ⊗ ˜ 2 j = 2 j (ω )1(1 v) 15 (1 v), (ω )1v 3 v , √ j = 0 − 2 ⊗ − − j − 0 − 1 2 ⊗ where v h1( 3 ) v ( 1) 3 h2( 3 ) v. Furthermore, ⊗ = − j 2 ⊗ j = − − j 52 j K2(1 v) ( 1) 9 (1 v), K2v ( 1) 9 v .

j 2 ∼ j 2 j j 2 j 2 0 0 Hence VL (τ )=(MT (τ )⊗Mt )⊕(WT (τ )⊗Wt ) as τ -twisted Mk ⊗Mt -modules for j = 1, 2, where

j 2 = { ∈ j 2 | ˜ 2 = 2 } MT (τ ) u VL (τ ) (ω )1u 3 u , j 2 = { ∈ j 2 | ˜ 2 = 1 } = WT (τ ) u VL (τ ) (ω )1u 15 u , j 1, 2.

j 2 j 2 As in the τ-twisted case, the MT (τ ), j = 0, 1, 2 are equivalent and the WT (τ ), 2 0 2 2 0 2 j = 0, 1, 2 are equivalent. Set MT (τ ) = MT (τ ) and WT (τ ) = WT (τ ). Then

j 2 ∼ 2 j 2 j (3-20) VL (τ ) = (MT (τ ) ⊗ Mt ) ⊕ (WT (τ ) ⊗ Wt ), j = 0, 1, 2,

2 0 0 as τ -twisted Mk ⊗ Mt -modules. 488 KENICHIRO TANABE AND HIROMICHI YAMADA

Remark 3.5. The weight-three vector K was denoted by different symbols in 3 previous papers, namely, vt , v , and q were used in [Dong et al. 2004], [Kitazume et al. 2003√], and [Kitazume√ et al.√ 2000b], respectively. They are related as follows: 3 K = −2 2vt = −2 2v = 2 2q. Thus, in the proof of [Kitazume et al. 2003, 3 j Proposition√ 6.8] (v )2 should act on the top level of VL (τ) as a scalar multiple of (−1) j /9 2 for j = 1, 2. Moreover, (6.46) of [Kitazume et al. 2003] and the j equation for VL (τ) on page 265 of [Dong et al. 2004] should be replaced with Equation (3-19). This correction does not affect the results in the latter paper. However, certain changes are necessary in [Kitazume et al. 2003] along with the correction. Note that

i i i ε MT (τ )(ε) = {u ∈ MT (τ ) | τ u = ξ u}, i i i ε WT (τ )(ε) = {u ∈ WT (τ ) | τ u = ξ u} for i = 1, 2, ε = 0, 1, 2. Another notation was used in [Kitazume et al. 2003], namely,

i ε M i i ε M i MT (τ ) = (MT (τ ))n, WT (τ ) = (WT (τ ))n, n∈1/9+ε/3+ޚ n∈2/45+ε/3+ޚ

1 where Un denotes the eigenspace of U with eigenvalue n for (ω˜ )1. The two sets of notation are related by

i ε i i ε i (3-21) MT (τ ) = MT (τ )(2ε), WT (τ ) = WT (τ )(2ε − 1).

Likewise, T T T T χ j ε = M χ j χ j 2 ε = M χ j 2 VL (τ) VL (τ) n, VL (τ ) VL (τ ) n n∈1/9+ε/3+ޚ n∈1/9+ε/3+ޚ of [Kitazume et al. 2003, (7.16)] are denoted here by

j ε j j 2 ε j 2 (3-22) (VL (τ)) = VL (τ)(2ε), (VL (τ )) = VL (τ )(2ε) for j = 0, 1, 2 and ε = 0, 1, 2, where Un is the eigenspace of U with eigenvalue n for ω1. 2 0 0 By (3-3), the minimal eigenvalues of (ω˜ )1 on Mt and Wt are 0 and 2/5, re- j j spectively, while those on Mt and Wt , j = 1, 2, are 2/3 and 1/15, respectively. Hence it follows from (3-19) that 0 ε ∼ ε 0 ε−1 0 (VL (τ)) = (MT (τ) ⊗ Mt ) ⊕ (WT (τ) ⊗ Wt ), (3-23) j ε ∼ ε+1 3− j ε 3− j (VL (τ)) = (MT (τ) ⊗ Mt ) ⊕ (WT (τ) ⊗ Wt ), j = 1, 2, ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 489

0 0 0 as M -modules for ε = 0, 1, 2, where M = M(0) ⊗ Mt . Similarly,

0 2 ε ∼ 2 ε 0 2 ε−1 0 (VL (τ )) = (MT (τ ) ⊗ Mt ) ⊕ (WT (τ ) ⊗ Wt ), (3-24) j 2 ε ∼ 2 ε+1 j 2 ε j (VL (τ )) = (MT (τ ) ⊗ Mt ) ⊕ (WT (τ ) ⊗ Wt ), j = 1, 2, as M0-modules for ε = 0, 1, 2 (see [Kitazume et al. 2003, (7.17)]). The following fusion rules of simple M(0)-modules will be necessary for the τ study of simple VL -modules.

c c W(0) × Mk = Wk , c c c W(0) × Wk = Mk + Wk , W(0) × M(ε) = W(ε), (3-25) W(0) × W(ε) = M(ε) + W(ε), i i W(0) × MT (τ )(ε) = WT (τ )(ε), i i i W(0) × WT (τ )(ε) = MT (τ )(ε) + WT (τ )(ε) for i = 1, 2 and ε = 0, 1, 2. In fact, the ﬁrst four fusion rules, that is, the fusion rules among simple M(0)-modules appearing in untwisted simple VL -modules, can be found in [Tanabe 2005]. The last two fusion rules involve simple M(0)-modules i that appear in τ -twisted simple VL -modules. Their proofs can be found in the Appendix. Fusion rules possess certain symmetries. Let Mi , i = 1, 2, 3 be modules for a vertex operator algebra V . Then by [Frenkel et al. 1993, Propositions 5.4.7 and 5.5.2]

M3 M3 (M2)0 dim I = dim I = dim I , V M1 M2 V M2 M1 V M1 (M3)0 where (Mi )0 is the contragredient module of Mi . Recall that the contragredient 0 module (U , YU 0 ) of a V -module (U, YU ) is deﬁned as follows. As a vector space 0 L ∗ U = n(Un) is the restricted dual of U and YU 0 ( · , z) is determined by

zL(1) −2 L(0) −1 hYU 0 (a, z)v, ui = hv, YU (e (−z ) a, z )ui for a ∈ V , u ∈ U, and v ∈ U 0. In our case M(0) is generated by the Virasoro element ω˜ 1 and the weight-three vector J. Moreover, hL1(0)v, ui = hv, L1(0)ui and hJ(0)v, ui = −hv, J(0)ui. Since the 20 simple M(0)-modules are distinguished by the action of L1(0) and J(0) on their top levels, we know from [Dong et al. 2004, Tables 1, 3, and 4] that 490 KENICHIRO TANABE AND HIROMICHI YAMADA the contragredient modules of the simple M(0)-modules are as follows. M(ε)0 =∼ M(2ε), W(ε)0 =∼ W(2ε), ε = 0, 1, 2, c 0 ∼ c c 0 ∼ c (Mk ) = Mk ,(Wk ) = Wk , 0 ∼ 2 0 ∼ 2 MT (τ)(ε) = MT (τ )(ε), WT (τ)(ε) = WT (τ )(ε), ε = 0, 1, 2 (see also [Dong et al. 1998a, Lemma 3.7] and [Tanabe 2005, Section 4.2]).

4. Structure of simple modules

τ 0 0 0 0 0 0 Recall that VL = VL (0) = M ⊕W with M = M(0)⊗ Mt and W = W(0)⊗Wt . τ In this section we study the structure of the 30 known simple VL -modules listed in Lemma 3.2. We discuss decompositions of these simple modules as modules for M0. Those decompositions have been obtained in [Kitazume et al. 2003]. We review them brieﬂy. (Some corrections are needed in that paper; see Remark 3.5.) τ A vector in a VL -module is said to be of weight h if it is an eigenvector for 1 2 L(0) = ω1 with eigenvalue h. We calculate the action of (ω˜ )1, (ω˜ )1, J2, K2, P1, τ (J1 P)2, and (K1 P)2 on the top levels of the 30 known simple VL -modules. Recall that the top level of a module means the homogeneous subspace of the module of smallest weight. The calculation is accomplished directly from the deﬁnition of untwisted or twisted vertex operators associated with the lattice L and the automorphisms τ and τ 2 (see [Dong and Lepowsky 1996; Frenkel et al. 1988; Lepowsky and Li 2004]). The results in this section will be used to determine the Zhu algebra τ τ A(VL ) of VL in Section 5. The vectors J1 P and K1 P are of weight 3. Their precise form in terms of the lattice vertex operator algebra VL is as follows. 3 2 2 3 J1 P = 2β1(−1) + 3β1(−1) β2(−1) − 3β1(−1)β2(−1) − 2β2(−1) − 4 (β2 − β0)(−1)x(α1) + (β0 − β1)(−1)x(α2) + (β1 − β2)(−1)x(α0) = 13 − 3 + − 2 − − − − 2 − − 3 − 9 2β1( 1) 3β1( 1) β2( 1) 3β1( 1)β2( 1) 2β2( 1) 4K, K1 P = 3 β1(−2)β2(−1) − β2(−2)β1(−1) − (β2 − β0)(−1)y(α1) + (β0 − β1)(−1)y(α2) + (β1 − β2)(−1)y(α0) = 7 − − − − − + 2 β1( 2)β2( 1) β2( 2)β1( 1) J. 0 0 0 The simple module VL(0). VL (0) = M ⊕ W as M -modules. The top level of 1 2 VL (0) is ރ1. By a property of the vacuum vector, all of (ω˜ )1, (ω˜ )1, J2, K2, P1, (J1 P)2, and (K1 P)2 act as 0 on ރ1.

The simple module VL(ε), ε = 1, 2. By (3-16), we have ∼ 0 0 VL (ε) = (M(ε) ⊗ Mt ) ⊕ (W(ε) ⊗ Wt ) ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 491

0 2,ε 2,ε as M -modules for ε = 1, 2. The top level of VL (ε) is ރv , where v = α1(−1)− ε 0 ξ α2(−1) ∈ W(ε) ⊗ Wt . We have √ 1 2,ε 3 2,ε 2 2,ε 2 2,ε 2,ε ε 2,ε (ω˜ )1v = v ,(ω˜ )1v = v , J2v = −(−1) 2 −3v , 5 5 √ 2,ε 2,ε 2,ε 2,ε ε 2,ε K2v = 0, P1v = 0,(J1 P)2v = 0,(K1 P)2v = (−1) 12 −3v .

The simple module VL(0, j) (0), j = 1, 2. For j = 1, 2, (3-4) implies that VL(0, j) is a direct sum of simple M0-modules of the form A ⊗ B, where A is a simple M(0)- 0 j j module and B is a simple Mt -module isomorphic to Mt or Wt . For convenience, j set U (ε) = VL(0, j) (ε), j = 1, 2, ε = 0, 1, 2. Let

j j j v3, j = e(−1) (β1−β2)/3 + e(−1) (β2−β0)/3 + e(−1) (β0−β1)/3.

3, j j 1 3, j 2 3, j 2 3, j 3, j Then v ∈U (0). Moreover, (ω )1v =(ω )1v =0 and (ω˜ )1v =(2/3)v . 3, j j j 0 j Hence v ∈ Mt and U (0) contains an Mt -submodule isomorphic to Mt . By j 0 j 0 the fusion rule Mt × Wt = Wt of Mt -modules and [Dong and Lepowsky 1996, j 0 j Proposition 11.9], U (0) contains an Mt -submodule isomorphic to Wt also. Thus j 0 j 0 j U (0) contains simple M -submodules of the form A⊗ Mt and A ⊗Wt for some simple M(0)-modules A and A0. The minimal weight of VL(0, j) is 2/3. Its weight subspace is of dimension 3 and j j j spanned by e(−1) (β1−β2)/3, e(−1) (β2−β0)/3, and e(−1) (β0−β1)/3. Thus the weight-2/3 j 3, j 1 3, j subspace of U (0) is ރv . Since (ω˜ )1v = 0 and since only M(0) is the simple 1 M(0)-module whose minimal weight (= eigenvalue of (ω˜ )1) is 0 by [Dong et al. 2004], we conclude that U j (0) contains a simple M0-submodule isomorphic to j M(0) ⊗ Mt . 2 j 1 The minimal eigenvalue of (ω˜ )1 in Wt is 1/15. Thus the eigenvalues of (ω˜ )1 on A0 must be of the form 3/5 + n, n ∈ ޚ. By [Dong et al. 2004], only W(0), W(1), W(2) are the simple M(0)-modules whose weights are of this form. The minimal weight of these simple modules are 8/5, 3/5 and 3/5, respectively. Since the weight-2/3 subspace of U j (0) is one dimensional, we see that U j (0) contains 0 j a simple M -submodule isomorphic to W(0) ⊗ Wt . 0 From the fusion rules for Mt -modules, we obtain the fusion rules

0 j j (M(ε) ⊗ Mt ) × (M(0) ⊗ Mt ) = M(ε) ⊗ Mt , 0 j j (W(ε) ⊗ Wt ) × (M(0) ⊗ Mt ) = W(ε) ⊗ Wt

0 j ∼ j j for M -modules. Hence U (ε) = (M(ε)⊗Mt )⊕(W(ε)⊗Wt ) for j = 1, 2 and ε = ∼ j j 0, 1, 2 by (3-4) and (3-16). In particular, VL(0, j) (0) = (M(0)⊗ Mt )⊕(W(0)⊗Wt ) 0 3, j j as M -modules, j = 1, 2. The top level of VL(0, j) (0) is ރv ⊂ M(0) ⊗ Mt . We have 492 KENICHIRO TANABE AND HIROMICHI YAMADA

˜ 1 3, j = ˜ 2 3, j = 2 3, j 3, j = 3, j = − − j 52 3, j (ω )1v 0,(ω )1v 3 v , J2v 0, K2v ( 1) 9 v , 3, j 3, j 3, j P1v = 0,(J1 P)2v = 0,(K1 P)2v = 0.

The simple module VL(0, j) (ε), j = 1, 2, ε = 1, 2. We have shown above that ∼ j j 0 VL(0, j) (ε) = (M(ε) ⊗ Mt ) ⊕ (W(ε) ⊗ Wt ) as M -modules, j = 1, 2, ε = 1, 2. The 4, j,ε top level of VL(0, j) (ε) is ރv , where j j j 4, j,ε (−1) (β1−β2)/3 2ε (−1) (β2−β0)/3 ε (−1) (β0−β1)/3 j v = e + ξ e + ξ e ∈ W(ε) ⊗ Wt . We have √ 1 4, j,ε 3 4, j,ε 2 4, j,ε 1 4, j,ε 4, j,ε ε 4, j,ε (ω˜ )1v = v ,(ω˜ )1v = v , J2v = −(−1) 2 −3v , 5 15 √ 4, j,ε j 2 4, j,ε 4, j,ε j+ε 4, j,ε K2v = (−1) v , P1v = −(−1) −3v , 9 √ 4, j,ε j 4, j,ε 4, j,ε ε 4, j,ε (J1 P)2v = −(−1) 24v ,(K1 P)2v = −(−1) 2 −3v . ∼ c 0 c 0 0 The simple module VL(c,0) . By (3-4), VL(c,0) = (Mk ⊗ Mt ) ⊕ (Wk ⊗ Wt ) as M - 5,1 5,2 modules. The top level of VL(c,0) is of dimension 2 with basis {v , v }, where 5,1 β1/2 −β1/2 c 0 5,2 β1/2 −β1/2 c 0 v = e − e ∈ Mk ⊗ Mt , v = e + e ∈ Wk ⊗ Wt . We have ˜ 1 5,1 = 1 5,1 ˜ 1 5,2 = 1 5,2 ˜ 2 5,1 = ˜ 2 5,2 = 2 5,2 (ω )1v 2 v ,(ω )1v 10 v ,(ω )1v 0,(ω )1v 5 v , 5, j 5, j 5,1 5,2 5,2 5,1 J2v = 0, K2v = 0, j = 1, 2, P1v = −v , P1v = v , 5, j 5, j (J1 P)2v = 0,(K1 P)2v = 0, j = 1, 2. ∼ The simple module VL(c, j) , j = 1, 2. By (3-4), we have the isomorphism VL(c, j) = c j c j 0 6, j (Mk ⊗ Mt )⊕(Wk ⊗Wt ) as M -modules, j = 1, 2. The top level of VL(c, j) is ރv , j 6, j −(−1) (β2−β0)/6 c j where v = e ∈ Wk ⊗ Wt . We have ˜ 1 6, j = 1 6, j ˜ 2 6, j = 1 6, j 6, j = 6, j = − j 2 6, j (ω )1v 10 v ,(ω )1v 15 v , J2v 0, K2v ( 1) 9 v , 6, j 6, j j 6, j 6, j P1v = 0,(J1 P)2v = (−1) 2v ,(K1 P)2v = 0. 0 0 ∼ The simple module VL(τ)(0). By (3-23), we have the isomorphism VL (τ)(0) = 0 0 0 0 (MT (τ)(0) ⊗ Mt ) ⊕ (WT (τ)(0) ⊗ Wt ) as M -modules. The top level of VL (τ)(0) 7 7 = ⊗ ∈ ⊗ 0 6= ∈ is ރv , where v 1 v MT (τ)(0) Mt and 0 v Tχ0 . We have √ ˜ 1 7 = 1 7 ˜ 2 7 = 7 = 14 − 7 7 = (ω )1v 9 v ,(ω )1v 0, J2v 81 3v , K2v 0, 7 7 7 P1v = 0,(J1 P)2v = 0,(K1 P)2v = 0. 0 0 ∼ The simple module VL(τ)(1). By (3-23), we have the isomorphism VL (τ)(1) = 0 0 0 0 (MT (τ)(1) ⊗ Mt ) ⊕ (WT (τ)(1) ⊗ Wt ) as M -modules. The top level of VL (τ)(1) is of dimension 2 with basis {v8,1, v8,2}, where 8,1 2 0 v = h2(−1/3) ⊗ v ∈ MT (τ)(1) ⊗ Mt , 8,2 0 v = h1(−2/3) ⊗ v ∈ WT (τ)(1) ⊗ Wt ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 493

6= ∈ and 0 v Tχ0 . We have √ 1 8,1 1 2 8,1 2 8,1 8,1 238 8,1 8,1 (ω˜ )1v = + v ,(ω˜ )1v = 0, J2v = − −3v , K2v = 0, 9 3 √ 81 8,1 = − 4 8,2 8,1 = 104 − 8,2 8,1 = P1v 3 v ,(J1 P)2v 9 3v ,(K1 P)2v 0, √ 1 8,2 2 1 8,2 2 8,2 2 8,2 8,2 22 8,2 8,2 (ω˜ )1v = + v ,(ω˜ )1v = v , J2v = − −3v , K2v = 0, 45 3 √5 81 √ 8,2 = 8,1 8,2 = − 52 − 8,1 8,2 = − 20 − 8,2 P1v 2v ,(J1 P)2v 3 3v ,(K1 P)2v 3 3v .

0 0 ∼ The simple module VL(τ)(2). By (3-23), we have the isomorphism VL (τ)(2) = 0 0 0 0 (MT (τ)(2) ⊗ Mt ) ⊕ (WT (τ)(2) ⊗ Wt ) as M -modules. The top level of VL (τ)(2) 9 9 = − ⊗ ∈ ⊗ 0 6= ∈ is ރv , where v h2( 1/3) v WT (τ)(2) Wt and 0 v Tχ0 . We have √ 1 9 2 9 2 9 2 9 9 4 9 9 (ω˜ )1v = v ,(ω˜ )1v = v , J2v = − −3v , K2v = 0, 45 5 81 √ 9 = 9 = 9 = 4 − 9 P1v 0,(J1 P)2v 0,(K1 P)2v 3 3v .

j The simple module VL (τ)(0), j = 1, 2. By (3-23), we have the isomorphism j ∼ 3− j 3− j 0 VL (τ)(0) = (MT (τ)(2)⊗Mt )⊕(WT (τ)(2)⊗Wt ) as M -modules for j = 1, 2. j 10, j 10, j 3− j The top level of VL (τ)(0) is ރv , where v = 1 ⊗ v ∈ WT (τ)(2) ⊗ Wt and 6= ∈ 0 v Tχ j . We have √ 1 10, j 2 10, j 2 10, j 1 10, j 10, j 4 10, j (ω˜ )1v = v ,(ω˜ )1v = v , J2v = − −3v , 45 15 √ 81 10, j j 2 10, j 10, j j 1 10, j K2v = −(−1) v , P1v = (−1) −3v , 9 9 √ 10, j = − j 8 10, j 10, j = − 2 − 10, j (J1 P)2v ( 1) 9 v ,(K1 P)2v 9 3v .

j The simple module VL (τ)(1), j = 1, 2. By (3-23), we have the isomorphism j ∼ 3− j 3− j 0 VL (τ)(1) = (MT (τ)(0)⊗Mt )⊕(WT (τ)(0)⊗Wt ) as M -modules for j = 1, 2. j 11, j,1 11, j,2 The top level of VL (τ)(1) is of dimension 2 with basis {v , v }, where √ 11, j,1 j 2 3− j v = h1(−2/3) ⊗ v − (−1) −3h2(−1/3) ⊗ v ∈ MT (τ)(0) ⊗ Mt , √ 11, j,2 j 2 3− j v = 2h1(−2/3) ⊗ v + (−1) −3h2(−1/3) ⊗ v ∈ WT (τ)(0) ⊗ Wt 6= ∈ and 0 v Tχ j . We have √ 1 11, j,1 1 11, j,1 2 11, j,1 2 11, j,1 11, j,1 14 11, j,1 (ω˜ )1v = v ,(ω˜ )1v = v , J2v = −3v , 9 3 √ 81 11, j,1 j 52 11, j,1 11, j,1 j 4 11, j,2 K2v = (−1) v , P1v = −(−1) −3v , 9 9 √ 11, j,1 = − j 52 11, j,2 11, j,1 = − 28 − 11, j,2 (J1 P)2v ( 1) 9 v ,(K1 P)2v 9 3v , ˜ 1 11, j,2 = 2 + 2 11, j,2 ˜ 2 11, j,2 = 1 11, j,2 (ω )1v 45 3 v ,(ω )1v 15 v , 494 KENICHIRO TANABE AND HIROMICHI YAMADA √ 11, j,2 176 11, j,2 11, j,2 j 2 11, j,2 J2v = −3v , K2v = −(−1) v , 81 √ √ 9 11, j,2 = − − j 8 − 11, j,1 + − j 5 − 11, j,2 P1v ( 1) 9 3v ( 1) 9 3v , 11, j,2 j 104 11, j,1 j 200 11, j,2 (J1 P)2v = (−1) v − (−1) v , √9 √9 11, j,2 = − 56 − 11, j,1 − 10 − 11, j,2 (K1 P)2v 9 3v 9 3v .

j The simple module VL (τ)(2), j = 1, 2. By (3-23), we have the isomorphism j ∼ 3− j 3− j 0 VL (τ)(2) = (MT (τ)(1)⊗Mt )⊕(WT (τ)(1)⊗Wt ) as M -modules for j = 1, 2. j 12, j 12, j The top level of VL (τ)(2) is ރv , where v = h2(−1/3) ⊗ v ∈ WT (τ)(1) ⊗ 3− j 6= ∈ Wt and 0 v Tχ j . We have √ 1 12, j 2 1 12, j 2 12, j 1 12, j 12, j 22 12, j (ω˜ )1v = + v ,(ω˜ )1v = v , J2v = − −3v , 45 3 15 √ 81 12, j j 2 12, j 12, j j 5 12, j K2v = −(−1) v , P1v = −(−1) −3v , 9 9 √ 12, j = − j 8 12, j 12, j = 10 − 12, j (J1 P)2v ( 1) 9 v ,(K1 P)2v 9 3v .

0 2 0 2 ∼ The simple module VL(τ )(0). By (3-24), we have the isomorphism VL (τ )(0) = 2 0 2 0 0 0 2 (MT (τ )(0)⊗Mt )⊕(WT (τ )(0)⊗Wt ) as M -modules. The top level of VL (τ )(0) 13 13 2 0 is ރv , where v = 1 ⊗ v ∈ MT (τ )(0) ⊗ M and 0 6= v ∈ T 0 . We have t χ0 √ ˜ 1 13 = 1 13 ˜ 2 13 = 13 = − 14 − 13 13 = (ω )1v 9 v ,(ω )1v 0, J2v 81 3v , K2v 0, 13 13 13 P1v = 0,(J1 P)2v = 0,(K1 P)2v = 0.

0 2 0 2 ∼ The simple module VL(τ )(1). By (3-24), we have the isomorphism VL (τ )(1) = 2 0 2 0 0 0 2 (MT (τ )(1)⊗Mt )⊕(WT (τ )(1)⊗Wt ) as M -modules. The top level of VL (τ )(1) is of dimension 2 with basis {v14,1, v14,2}, where

14,1 0 2 2 0 v = h2(−1/3) ⊗ v ∈ MT (τ )(1) ⊗ Mt , 14,2 0 2 0 v = h1(−2/3) ⊗ v ∈ WT (τ )(1) ⊗ Wt , and 0 6= v ∈ T 0 . We have χ0 1 14,1 1 2 14,1 2 14,1 (ω˜ )1v = + v ,(ω˜ )1v = 0, 9√ 3 14,1 238 14,1 14,1 J2v = −3v , K2v = 0, 81 √ 14,1 = − 4 14,2 14,1 = − 104 − 14,2 14,1 = P1v 3 v ,(J1 P)2v 9 3v ,(K1 P)2v 0, 1 14,2 2 1 14,2 2 14,2 2 14,2 (ω˜ )1v = + v ,(ω˜ )1v = v , 45 √3 5 14,2 22 14,2 14,2 J2v = −3v , K2v = 0, 81 √ √ 14,2 = 14,1 14,2 = 52 − 14,1 14,2 = 20 − 14,2 P1v 2v ,(J1 P)2v 3 3v ,(K1 P)2v 3 3v . ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 495

0 2 The simple module VL(τ )(2). By Equation (3-24), we have the isomorphism 0 2 ∼ 2 0 2 0 0 VL (τ )(2) = (MT (τ )(2) ⊗ Mt ) ⊕ (WT (τ )(2) ⊗ Wt ) as M -modules. The top 0 2 15 15 = 0 − ⊗ ∈ 2 ⊗ 0 level of VL (τ )(2) is ރv , where v h2( 1/3) v WT (τ )(2) Wt and 0 6= v ∈ Tχ0 . We have 0 √ 1 15 2 15 2 15 2 15 15 4 15 15 (ω˜ )1v = v ,(ω˜ )1v = v , J2v = −3v , K2v = 0, 45 5 81 √ 15 = 15 = 15 = − 4 − 15 P1v 0,(J1 P)2v 0,(K1 P)2v 3 3v .

j 2 The simple module VL (τ )(0), j = 1, 2. By (3-24), we have the isomorphism j 2 ∼ 2 j 2 j 0 VL (τ )(0) = (MT (τ )(2)⊗ Mt )⊕(WT (τ )(2)⊗Wt ) as M -modules for j = 1, 2. j 2 16, j 16, j 2 j The top level of VL (τ )(0) is ރv , where v = 1 ⊗ v ∈ WT (τ )(2) ⊗ Wt and 0 6= v ∈ T 0 . We have χ j √ 1 16, j 2 16, j 2 16, j 1 16, j 16, j 4 16, j (ω˜ )1v = v ,(ω˜ )1v = v , J2v = −3v , 45 15 √ 81 16, j j 2 16, j 16, j j 1 16, j K2v = (−1) v , P1v = (−1) −3v , 9 9 √ 16, j = − − j 8 16, j 16, j = 2 − 16, j (J1 P)2v ( 1) 9 v ,(K1 P)2v 9 3v .

j 2 The simple module VL (τ )(1), j = 1, 2. By (3-24), we have the isomorphism j 2 ∼ 2 j 2 j 0 VL (τ )(1) = (MT (τ )(0)⊗ Mt )⊕(WT (τ )(0)⊗Wt ) as M -modules for j = 1, 2. j 2 17, j,1 17, j,2 The top level of VL (τ )(1) is of dimension 2 with basis {v , v }, where √ 17, j,1 0 j 0 2 2 j v = h1(−2/3) ⊗ v − (−1) −3h2(−1/3) ⊗ v ∈ MT (τ )(0) ⊗ Mt , √ 17, j,2 0 j 0 2 2 j v = 2h1(−2/3) ⊗ v + (−1) −3h2(−1/3) ⊗ v ∈ WT (τ )(0) ⊗ Wt and 0 6= v ∈ T 0 . We have χ j √ 1 17, j,1 1 17, j,1 2 17, j,1 2 17, j,1 17, j,1 14 17, j,1 (ω˜ )1v = v ,(ω˜ )1v = v , J2v = − −3v , 9 3 √ 81 17, j,1 j 52 17, j,1 17, j,1 j 4 17, j,2 K2v = −(−1) v , P1v = −(−1) −3v , 9 9 √ 17, j,1 = − − j 52 17, j,2 17, j,1 = 28 − 17, j,2 (J1 P)2v ( 1) 9 v ,(K1 P)2v 9 3v , 1 17, j,2 2 2 17, j,2 2 17, j,2 1 17, j,2 (ω˜ )1v = + v ,(ω˜ )1v = v , √45 3 15 17, j,2 176 17, j,2 17, j,2 j 2 17, j,2 J2v = − −3v , K2v = (−1) v , 81 √ √ 9 17, j,2 = − − j 8 − 17, j,1 + − j 5 − 17, j,2 P1v ( 1) 9 3v ( 1) 9 3v , 17, j,2 j 104 17, j,1 j 200 17, j,2 (J1 P)2v = −(−1) v + (−1) v , √ 9 √ 9 17, j,2 = 56 − 17, j,1 + 10 − 17, j,2 (K1 P)2v 9 3v 9 3v .

j 2 The simple module VL (τ )(2), j = 1, 2. By (3-24), we have the isomorphism j 2 ∼ 2 j 2 j 0 VL (τ )(2) = (MT (τ )(1)⊗ Mt )⊕(WT (τ )(1)⊗Wt ) as M -modules for j = 1, 2. 496 KENICHIRO TANABE AND HIROMICHI YAMADA

j 2 18, j 18, j = 0 − ⊗ ∈ 2 ⊗ j The top level of VL (τ )(2) is ރv , where v h2( 1/3) v WT (τ )(1) Wt and 0 6= v ∈ Tχ0 . We have j √ 1 18, j 2 1 18, j 2 18, j 1 18, j 18, j 22 18, j (ω˜ )1v = + v ,(ω˜ )1v = v , J2v = −3v , 45 3 15 √ 81 18, j j 2 18, j 18, j j 5 18, j K2v = (−1) v , P1v = −(−1) −3v , 9 9 √ 18, j = − − j 8 18, j 18, j = − 10 − 18, j (J1 P)2v ( 1) 9 v ,(K1 P)2v 9 3v .

Symmetries by σ . We now consider the automorphisms σ and θ of VL that are lifts of the isometries σ and θ of the lattice L deﬁned by (3-1). Clearly, σ τσ = τ 2, τ σ θ = θσ , and τθ = θτ. Thus σ and θ induce automorphisms of VL of order 2. We have σ J = −J, σ K = −K , σ P = P, θ J = J, θ K = −K , and θ P = −P. Hence 0 σ and θ induce the same automorphism of Mt and θ is the identity on M(0). Note also that σ (J1 P) = −J1 P and σ (K1 P) = −K1 P. τ From the action of σ on the top level of the 30 known simple VL -modules or the action of J2, K2, (J1 P)2, and (K1 P)2, we know how σ permutes those simple τ τ VL -modules. In fact, σ transforms VL(c,0) into an equivalent simple VL -module and τ interchanges the remaining simple VL -modules as follows.

VL (1) ↔ VL (2), VL(0,1) (ε) ↔ VL(0,2) (2ε), ε = 0, 1, 2, j j 2 VL(c,1) ↔ VL(c,2) , VL (τ)(ε) ↔ VL (τ )(ε), j, ε = 0, 1, 2.

3−i 0 j 2 Note that σ hi = ξ hi , i = 1, 2. The top level of VL (τ )(ε) can be obtained 0 j by replacing hi (i/3 + n) with hi (i/3 + n) in the top level of VL (τ)(ε) for j, ε = 0, 1, 2. The corresponding action of σ on the simple M(0)-modules was discussed in [Dong et al. 2004, Section 4.4].

5. Classiﬁcation of simple modules

τ 0 0 0 We keep the notation in the preceding section. Thus VL = M ⊕ W with M = 0 0 0 τ M(0) ⊗ Mt and W = W(0) ⊗ Wt . In this section we show that any simple VL - τ module is equivalent to one of the 30 simple VL -modules listed in Lemma 3.2. The τ τ result will be established by considering the Zhu algebra A(VL ) of VL . First, we review some notation and basic formulas for the Zhu algebra A(V ) of a vertex operator algebra (V, Y, 1, ω). Deﬁne two binary operations ∞ ∞ X wt u X wt u (5-1) u ∗ v = u − v, u ◦ v = u − v i i 1 i i 2 i=0 i=0 for u, v ∈ V with u being homogeneous and extend ∗ and ◦ for arbitrary u ∈ V by linearity. Let O(V ) be the subspace of V spanned by all u ◦ v for u, v ∈ V . Set A(V ) = V/O(V ). By [Zhu 1996, Theorem 2.1.1], O(V ) is a two-sided ideal with ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 497 respect to the operation ∗. Thus ∗ induces an operation in A(V ). Denote by [v] the image of v ∈ V in A(V ). Then [u] ∗ [v] = [u ∗ v] and A(V ) is an associative algebra by this operation. Moreover, [1] is the identity and [ω] is in the center of A(V ). For u, v ∈ V , we write u ∼ v if [u] = [v]. For ϕ, ψ ∈ End V , we write ϕ ∼ ψ if ϕv ∼ ψv for all v ∈ V . We need some basic formulas from [Zhu 1996]. ∞ X wt(u) − 1 (5-2) v ∗ u ∼ u − v, i i 1 i=0 ∞ X wt(u) + m (5-3) u − − v ∈ O(V ), n ≥ m ≥ 0. i i n 2 i=0 Moreover (see [Wang 1993]), (5-4) L(−n) ∼ (−1)n(n − 1)L(−2) + L(−1) + L(0) , n ≥ 1,

(5-5) [ω] ∗ [u] = [(L(−2) + L(−1))u], where L(n) = ωn+1. From (5-4) and (5-5) we have (5-6) [L(−n)u] = (−1)n(n − 1)[ω] ∗ [u] + (−1)n[L(0)u], n ≥ 1. If u ∈ V is of weight 2, then u(−n −3)+2u(−n −2)+u(−n −1) ∼ 0 by (5-3), where u(n) = un+1. Hence (5-7) u(−n) ∼ (−1)n(n − 1)u(−2) + (n − 2)u(−1) for n ≥ 1. Then it follows from (5-1) and (5-2) that (5-8) u(−n)w ∼ (−1)n − u ∗ w + nw ∗ u + u(0)w for n ≥ 1, w ∈ V . Likewise, if u is of weight 3 and u(n) = un+2, then (5-9) u(−n) ∼ (−1)n+1 · 1 − − − + − − − + 1 − − − 2 (n 1)(n 2)u( 3) (n 1)(n 3)u( 2) 2 (n 2)(n 3)u( 1) , − ∼ − n+1 − + − − − ∗ + 1 − ∗ (5-10) u( n)w ( 1) nu( 1)w (n 1)u(0)w (n 1)u w 2 n(n 1)w u , for n ≥ 1, w ∈ V . For a homogeneous vector u ∈ V , o(u) = uwt(u)−1 is the weight zero component operator of Y (u, z). Extend o(u) for arbitrary u ∈ V by linearity. Note that we call a module in the sense of [Zhu 1996] an ގ-graded weak module here. If M = ⊕∞ 6= n=0 M(n) is an ގ-graded weak V -module with M(0) 0, then o(u) acts on its top level M(0). Zhu’s theory [1996] says: (1) o(u)o(v) = o(u ∗ v) as operators on the top level M(0) and o(u) acts as 0 on M(0) if u ∈ O(V ). Thus M(0) is an A(V )- module, where [u] acts on M(0) as o(u). (2) The map M 7→ M(0) is a bijection 498 KENICHIRO TANABE AND HIROMICHI YAMADA between the set of isomorphism classes of simple ގ-graded weak V -modules and the set of isomorphism classes of simple A(V )-modules. τ i i We return to VL . As in Section 3, we write L (n) = (ω˜ )n+1, i = 1, 2, J(n) = 0 Jn+2, and K (n) = Kn+2. The Zhu algebras A(M(0)) and A(Mt ) were determined in [Dong et al. 2004] and [Kitazume et al. 2000b], respectively. Since 0 τ 0 τ O(M ) ⊂ O(VL ), the image of M(0) (resp. Mt ) in A(VL ) is a homomorphic 0 1 2 image of A(M(0)) (resp. A(Mt )). It is generated by [ω ˜ ], [J] (resp. [ω ˜ ], [K ]). By a direct calculation, we have

1 2 P1 P = −16ω˜ − 6ω˜ , 1 2 P0 P = −8(ω˜ )−21 − 3(ω˜ )−21, = 5 − 12 ˜ 1 − 18 ˜ 2 P−1 P 273 J1 K1 P 7 (ω )−31 13 (ω )−31 − 36 ˜ 1 ˜ 1 − 9 ˜ 2 ˜ 2 − ˜ 1 ˜ 2 (5-11) 7 (ω )−1(ω )−11 13 (ω )−1(ω )−11 16(ω )−1(ω )−11, = 1 + 1 − 8 ˜ 1 − 12 ˜ 2 − 36 ˜ 1 ˜ 1 P−2 P 84 J0 K1 P 156 J1 K0 P 7 (ω )−41 13 (ω )−41 7 (ω )−2(ω )−11 − 9 ˜ 2 ˜ 2 − ˜ 1 ˜ 2 − ˜ 1 ˜ 2 13 (ω )−2(ω )−11 8(ω )−2(ω )−11 8(ω )−1(ω )−21.

Moreover, J2 P = K2 P = 0. Then, using formulas (5-4)–(5-10), we obtain [ ] ∗ [ ] = 5 [ ] − 36 [ ˜ 1] ∗ [ ˜ 1] − 9 [ ˜ 2] ∗ [ ˜ 2] (5-12) P P 273 J1 K1 P 7 ω ω 13 ω ω − [ ˜ 1] ∗ [ ˜ 2] + 4 [ ˜ 1] + 6 [ ˜ 2] 16 ω ω 7 ω 13 ω ,

[ ◦ ] = 1 [ ]∗[ ]− 1 [ ]∗[ ]+ 1 [ ]∗[ ]− 1 [ ]∗[ ] (5-13) P P 84 J K1 P 84 K1 P J 156 K J1 P 156 J1 P K = 0.

τ 1 2 It turns out that A(VL ) is generated by [ω ˜ ], [ω ˜ ], [J], [K ], and [P] (Corollary 5.11). However, we ﬁrst prove the following intermediate assertion.

τ 1 2 Proposition 5.1. The Zhu algebra A(VL ) is generated by [ω ˜ ], [ω ˜ ], [J], [K ], [P], [J1 P], and [K1 P]. i = = ≥ 1 = 8 2 = 2 Proof. Recall that L (n)P 0 for i 1, 2, n 1, L (0)P 5 P, L (0)P 5 P, and J(n)P = K (n)P = 0 for n ≥ 0. Thus from the commutation relations (3-9)–(3-11) and (3-13)–(3-15) we see that W 0 is spanned by the vectors of the form

1 1 2 2 (5-14) L (− j1) ··· L (− jr )L (−k1) ··· L (−ks)

· J(−m1) ··· J(−m p)K (−n1) ··· K (−nq )P with j1 ≥ · · · ≥ jr ≥ 1, k1 ≥ · · · ≥ ks ≥ 1, m1 ≥ · · · ≥ m p ≥ 1, n1 ≥ · · · ≥ nq ≥ 1. Let v be a vector of this form. Its weight is

j1 + · · · + jr + k1 + · · · + ks + m1 + · · · + m p + n1 + · · · + nq + 2. ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 499

τ 0 0 0 τ 1 Since VL = M ⊕W and since the image of M in A(VL ) is generated by [ω ˜ ], 2 τ [ω ˜ ], [J], and [K ], it sufﬁces to show that the image [v] of v in A(VL ) is contained 1 2 in the subalgebra generated by [ω ˜ ], [ω ˜ ], [J], [K ], [P], [J1 P], and [K1 P]. We proceed by induction on the weight of v. By formula (5-8) with u =ω ˜ i , i = 1, 2 and the induction on the weight, we may assume that r = s = 0, that is,

v = J(−m1) ··· J(−m p)K (−n1) ··· K (−nq )P.

Moreover, by formula (5-10) with u = J, we may assume that m1 =· · ·=m p =1. Since J(m) and K (n) commute, we may also assume that n1 = · · · = nq = 1 by a similar argument. Then v = J(−1)p K (−1)q P. Next, we reduce v to the case p ≤ 1. For this purpose, we use a singular vector (5-15) 5J(−1)2 P + 2496L1(−2)P − 195L1(−1)2 P = 0. in W(0). Suppose p ≥ 2. Then, since K (−1) commutes with J(m) and L1(n), (5-15) implies that v = J(−1)p K (−1)q P is a linear combination of J(−1)p−2 L1(−2)K (−1)q P and J(−1)p−2 L1(−1)2 K (−1)q P. By (3-10), these two vectors can be written in the form L1(−2)HK (−1)q P and L1(−1)2 H 0 K (−1)q P, where H (resp. H 0) is a polynomial in J(−1) and J(−3) (resp. J(−1), J(−2), and J(−3)). Then by (5-8) with u =ω ˜ 1 and the induction on the weight, the assertion holds for v. Hence we may assume that p ≤ 1. There is a singular vector (5-16) K (−1)2 P − 210L2(−2)P = 0

0 in Wt . Thus, by a similar argument as above, we may assume that q ≤ 1. Finally, it follows from (5-12) that [J(−1)K (−1)P] can be written by [ω ˜ 1], [ω ˜ 2], and [P] τ in A(VL ). The proof is complete. τ We will classify the simple VL -modules using our knowledge of simple modules 0 for M(0) and Mt together with fusion rules (3-25) and (3-7). Set c i ᏹ1 = {M(ε), Mk , MT (τ )(ε) | i = 1, 2, ε = 0, 1, 2}, c i ᐃ1 = {W(ε), Wk , WT (τ )(ε) | i = 1, 2, ε = 0, 1, 2}, j j ᏹ2 = {Mt | j = 0, 1, 2}, ᐃ2 = {Wt | j = 0, 1, 2}.

Then ᏹ1 ∪ ᐃ1 (resp. ᏹ2 ∪ ᐃ2) is a complete set of representatives of isomor- 0 phism classes of simple M(0)-modules (resp. simple Mt -modules). A main point is that the fusion rules of the following form hold. W(0) × M1 = W 1, W(0) × W 1 = M1 + W 1, (5-17) 0 2 2 0 2 2 2 Wt × M = W , Wt × W = M + W , 500 KENICHIRO TANABE AND HIROMICHI YAMADA

i i i where M ∈ ᏹi , i = 1, 2, and W ∈ ᐃi is determined by M through the fusion 1 1 0 2 2 rule W(0) × M = W or Wt × M = W . 0 Recall that M is rational, C2-cofinite, and of CFT type. Thus every ގ-graded weak M0-module is a direct sum of simple M0-modules. As a result, every ގ- τ 0 graded weak VL -module is decomposed into a direct sum of simple M -modules, and in particular L(0) = ω1 acts semisimply on it. Each weight subspace, that is, each eigenspace for L(0) is not necessarily a finite dimensional space. However, τ τ any simple weak VL -module is a simple ordinary VL -module by [Abe et al. 2004, τ Corollary 5.8], since VL is C2-cofinite and of CFT type. We note that 0 0 τ (5-18) W · W = VL . 0 0 0 0 τ Indeed, W ·W = span{anb | a, b ∈ W , n ∈ ޚ} is an M -submodule of VL by (2-6). 0 1 2 0 0 0 0 0 Since P, J1 K1 P ∈ W and ω˜ , ω˜ ∈ M , (5-11) implies that W · W = M ⊕ W . Each simple M0-module is isomorphic to a tensor product A ⊗ B of a simple 0 M(0)-module A and a simple Mt -module B. We show that only restricted simple 0 τ M -modules can appear in ގ-graded weak VL -modules. τ 0 Lemma 5.2. Let U be an ގ-graded weak VL -module. Then any simple M - 1 2 1 2 i submodule of U is isomorphic to M ⊗ M or W ⊗ W for some M ∈ ᏹi and i W ∈ ᐃi , i = 1, 2. 0 0 ∼ 1 2 1 Proof. Suppose U contains a simple M -submodule S = M ⊗ W with M ∈ ᏹ1 2 τ 0 τ 0 and W ∈ ᐃ2. Let S = VL · S = span{anw | a ∈ VL , w ∈ S , n ∈ ޚ}. Then (2-6) τ 0 implies that S is the ގ-graded weak VL -submodule of U generated by S . By the construction of S, the difference of any two eigenvalues of L(0) in S is an integer. τ In fact, S is an ordinary VL -module by Remark 2.16. τ 0 If v is a nonzero vector in VL , then vn S 6= 0 for some n ∈ ޚ. Indeed, Lemma τ 0 τ 2.6 implies that the set {v ∈ VL | vn S = 0 for all n ∈ ޚ} is an ideal of VL . It is in τ 0 0 fact 0, since VL is a simple vertex operator algebra and S is a simple M -module. Then by the fusion rules (5-17), a simple M0-module isomorphic to W 1 ⊗ M2 or W 1 ⊗W 2 must appear in S. However, the difference of the minimal eigenvalues of L(0) in M1⊗W 2 and W 1⊗M2, or in M1⊗W 2 and W 1⊗W 2 is not an integer. This is a contradiction. Thus U does not contain a simple M0-submodule isomorphic to M1 ⊗ W 2. By a similar argument, we can also show that there is no simple 0 1 2 M -submodule isomorphic to W ⊗ M in U. Hence the assertion holds. 1 2 i 1 2 i Set ᏹ = {M ⊗ M | M ∈ ᏹi , i = 1, 2} and ᐃ = {W ⊗W | W ∈ ᐃi , i = 1, 2}. Then each of ᏹ and ᐃ consists of 30 inequivalent simple M0-modules. The top level of every simple M0-module is of dimension one. τ Lemma 5.3. If U is a simple ގ-graded weak VL -module whose top level is of τ dimension one, then U is isomorphic to one of the 23 known simple VL -modules ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 501 with one dimensional top level, namely, VL(0, j) (ε), j = 0, 1, 2, ε = 0, 1, 2, VL(c, j) , j j 2 j = 1, 2, VL (τ)(ε), j = 0, 1, 2, ε = 0, 2, and VL (τ )(ε), j = 0, 1, 2, ε = 0, 2. Proof. Since U is a direct sum of simple M0-modules and since the top level, say Uλ of U is assumed to be of dimension one, it follows from Lemma 5.2 that 1 2 1 2 Uλ is isomorphic to the top level of M ⊗ M or the top level of W ⊗ W as an 0 i i A(M )-module for some M ∈ ᏹi , W ∈ ᐃi , i = 1, 2. The Zhu algebra 0 ∼ 0 A(M ) = A(M(0)) × A(Mt ) is commutative and the action of A(M0) on the top level of M1 ⊗ M2 and the top level of W 1 ⊗ W 2 are known. Indeed, we know all possible action of the elements 1 2 τ 1 2 [ω ˜ ], [ω ˜ ], [J], and [K ] of A(VL ) on Uλ. Let [ω ˜ ], [ω ˜ ], [J], and [K ] act on Uλ as scalars a1, a2, b1, and b2, respectively. There are 60 possible such quadruplets (a1, a2, b1, b2). Let [P], [J1 P], and [K1 P] act on Uλ as scalars x1, x2, and x3, respectively. Then it follows from (5-12) that [J1 K1 P] acts on Uλ as a scalar 273 2 + 1404 2 + 189 2 + 4368 − 156 − 126 (5-19) 5 x1 5 a1 5 a2 5 a1a2 5 a1 5 a2. From computer calculations, whose results are presented in an online supple- ment to this paper,1 and from formulas (5-4)–(5-10), we conclude that the vanishing of [P ◦ (J1 P)] and [P ◦ (K1 P)] imply, respectively,

(5-20) 15b2x1 + 5a2x3 − 2x3 = 0,(15a2 − 1)x2 = 0. Using (5-19), we can calculate

[(J1 P) ∗ (J1 P)], [(K1 P) ∗ (K1 P)], [(J1 P) ∗ (K1 P)] in a similar way and verify that the following equations hold. 2 = 229164 − 37856 + 1669382 2 − 56 − 4056 + 348994464 3 (5-21) x2 575 a1 425 a2 48875 x1 85 b2x2 115 b1x3 107525 a1 + 137149584 2 − 1030224 2 + 7064876 2 − 40788488 9775 a1a2 1375 a1 9775 a1a2 48875 a1a2 + 16160456 − 419184 3 − 200994 2 + 1065516 − 3042 2 537625 a1 9775 a2 48875 a2 48875 a2 187 b1,

2 = − 37044 − 5684 + 741713 2 (5-22) x3 575 a1 85 a2 97750 x1 + 28 + 216 − 54559344 3− 28217448 2 + 254982 2− 25042724 2 221 b2x2 115 b1x3 107525 a1 9775 a1a2 1375 a1 9775 a1a2 + 26308184 − 8127098 − 4775148 3 + 188338017 2 − 9722139 − 180 2 48875 a1a2 537625 a1 25415 a2 1270750 a2 635375 a2 187 b1, (5-23) = − 864 2 + 1248 + 1152 2 + 5904 + 184176 − 62112 − x2x3 5 a1 25 a1a2 5 a2 125 a1 125 a2 625 x1 36b1b2.

1The authors can supply these expressions in machine readable form upon request. 502 KENICHIRO TANABE AND HIROMICHI YAMADA

We have obtained a system of equations (5-20)–(5-23) for x1, x2, x3. We can solve this system of equations with respect to the 60 possible quadruplets (a1, a2, b1, b2). Actually, there is no solution for 37 quadruplets of (a1, a2, b1, b2). For each of the remaining 23 quadruplets (a1, a2, b1, b2), the system of equations possesses a unique solution (x1, x2, x3). Furthermore, the 23 sets (a1, a2, b1, b2, 1 2 x1, x2, x3) of values determined in this way coincide with the action of [ω ˜ ], [ω ˜ ], τ [J], [K ], [P], [J1 P], and [K1 P] on the top level of the 23 known simple VL - τ modules with one dimensional top level described in Section 4. Since A(VL ) is generated by these seven elements, this implies that Uλ is isomorphic to the top τ τ level of one of the 23 simple VL -modules listed in the assertion as an A(VL )- module. Thus the lemma holds by Zhu’s theorem.

Remark 5.4. We also obtain some equations for x1x2 and x1x3 from [P ∗ (J1 P)] and [P ∗ (K1 P)]. However, they are not sufﬁcient to determine x1, x2, and x3. τ 0 Lemma 5.5. Every ގ-graded weak VL -module contains a simple M -submodule isomorphic to a member of ᏹ. τ Proof. Suppose false and let U be an ގ-graded weak VL -module which contains no simple M0-submodule isomorphic to a member of ᏹ. Then by Lemma 5.2, there 0 ∼ 1 2 i is a simple M -submodule W in U such that W = W ⊗ W for some W ∈ ᐃi , i = 1, 2. The top level of W, say Wλ for some λ ∈ ޑ, is a one dimensional space. τ τ Take 0 6= w ∈ Wλ and let S = VL · w = span{anw | a ∈ VL , n ∈ ޚ}, which is an τ τ 0 0 ordinary VL -module by (2-6) and Remark 2.16. Since VL = M ⊕ W , it follows from our assumption and the fusion rules (5-17) that S is isomorphic to a direct sum of ﬁnite number of copies of W as an M0-module. Thus [ω ˜ 1], [ω ˜ 2], [J], and [K ] act on the top level Sλ of S as scalars, say a1, a2, b1, and b2, respectively. Then by a similar calculation as in the proof of Lemma 5.3, we see that [P ◦(K1 P)] = 0 implies

(5-24) (15a2 − 1)o(J1 P) = 0

τ as an operator on the top level Sλ. Recall that [u] ∈ A(VL ) acts on Sλ as o(u) = τ uwt(u)−1 for a homogeneous vector u of VL . Furthermore, we can calculate that

o(J1 P)o(P) − o(P)o(J1 P) = 0, − = 2 − (5-25) o(K1 P)o(P) o(P)o(K1 P) 13 (15a2 1)o(J1 P), − = 96 − + + o(J1 P)o(K1 P) o(K1 P)o(J1 P) 125 (15a2 1)(65a1 100a2 441)o(P) as operators on Sλ. By (5-24), 15a2 − 1 = 0 or o(J1 P) = 0 and so o(P), o(J1 P), and o(K1 P) τ commute each other. Thus the action of A(VL ) on Sλ is commutative. Hence we τ can choose a one dimensional A(VL )-submodule T of Sλ. Zhu’s theory tells us that ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 503

τ there is a simple ގ-graded weak VL -module R whose top level Rλ is isomorphic τ to T as an A(VL )-module. Since dim Rλ = 1, R is isomorphic to one of the 23 τ 0 simple VL -modules listed in Lemma 5.3. In particular, R contains a simple M - τ submodule M isomorphic to a member of ᏹ. Now, consider the VL -submodule τ VL ·T of S generated by T . By Lemma 2.10, there is a surjective homomorphism of τ τ τ 0 VL -modules from VL ·T onto R. Then VL ·T must contain a simple M -submodule isomorphic to M. This contradicts our assumption. The proof is complete. τ 0 Lemma 5.6. Let U be an ގ-graded weak VL -module and M be a simple M - ∼ 1 2 0 i submodule of U such that M = M ⊗M as M -modules for some M ∈ᏹi , i =1, 2. τ τ τ Then VL · M = span{anu | a ∈ VL , u ∈ M, n ∈ ޚ} is a simple VL -module. Moreover, τ 0 1 2 VL · M = M ⊕ W, where W is a simple M -module isomorphic to W ⊗ W and W i , i = 1, 2 are determined from Mi by the fusion rules W(0) × M1 = W 1 and 0 2 2 Wt × M = W of (5-17). τ τ τ Proof. By Remark 2.16, VL · M is an ordinary VL -module. Note that VL · M = (M0 + W 0) · M = M + W 0 · M. We see that W 0 · M 6= 0 by a similar argument as 0 0 0 0 τ in the proof of Lemma 5.2. Actually, W · (W · M) ⊃ (W · W ) · M = VL · M (see Lemma 2.6 and (5-18)) implies W 0 · M 6= 0 also. Moreover, W 0 · M is an M0- module by (2-6). Since M0 is rational, W 0 · M is decomposed into a direct sum of 0 0 L γ 1 2 i simple M -modules, say W · M = γ ∈0 S . Let W = W ⊗W , where W ∈ ᐃi , 1 1 0 2 2 i = 1, 2 are determined by the fusion rules W(0)× M = W and Wt × M = W . W W The space IM0 W 0 M of intertwining operators of type W 0 M is of dimension one and each Sγ is isomorphic to W. We want to show that |0| = 1. Suppose 0 contains at least two elements and γ γ 0 take γ1, γ2 ∈ 0, γ1 6= γ2. Let ψ : S 2 → S 1 be an isomorphism of M -modules 0 γ 0 and pγ : W · M → S be a projection. For a ∈ W and u ∈ M, set = = Ᏻγ1 (a, z)u pγ1 YU (a, z)u, Ᏻγ2 (a, z)u ψpγ2 YU (a, z)u, τ where YU (a, z) is the vertex operator of the ގ-graded weak VL -module U. Then · = W Ᏻγi ( , z), i 1, 2 are nonzero members in the one dimensional space IM0 W 0 M , · = · 6= ∈ 6= ∈ γ1 so that µᏳγ1 ( , z) Ᏻγ2 ( , z) for some 0 µ ރ. Let 0 v S . Then ∈ 0 · = P j j j ∈ 0 j ∈ ∈ v W M and so v j (a )n j u for some a W , u M, n j ޚ. Take −n j −1 j j = j j the coefﬁcients of z in both sides of µᏳγ1 (a , z)u Ᏻγ2 (a , z)u . Then j j = j j µpγ1 ((a )n j u ) ψpγ2 ((a )n j u ). Summing up both sides of the equation with = ∈ γ1 = respect to j, we have µpγ1 v ψpγ2 v. However, v S implies that pγ1 v v = 6= 6= | | = and pγ2 v 0. This is a contradiction since µ 0 and v 0. Thus 0 1 and W 0 · M =∼ W as required. τ τ τ If VL · M is not a simple VL -module, then there is a proper VL -submodule N τ 0 of VL · M. Since M and W are simple M -modules, N must be isomorphic to M or W as an M0-module. Then the top level of N is of dimension one. The simple 504 KENICHIRO TANABE AND HIROMICHI YAMADA

τ VL -modules with one dimensional top level are classified in Lemma 5.3. Each of them is a direct sum of two simple M0-modules. However, N is not of such a form. τ τ Thus VL · M is a simple VL -module. Lemma 5.7. Let U = M ⊕ W be an M0-module such that M =∼ M1 ⊗ M2 and ∼ 1 2 i i W = W ⊗ W for some M ∈ ᏹi and W ∈ ᐃi , i = 1, 2. Then U admits at most τ one simple VL -module structure. τ Proof. Assume that (U, Y1) and (U, Y2) are simple VL -modules such that Yi (a, z)= Y (a, z) for all a ∈ M0, i = 1, 2, where (U, Y ) is the given M0-module structure. τ ˜ τ τ 0 We denote the vertex operator of VL by Y (v, z) for v ∈ VL . Let pM0 : VL → M τ 0 and pW 0 : VL → W be projections and define Ᏽ( · , z) and ᏶( · , z) by ˜ ˜ Ᏽ(a, z)b = pM0 Y (a, z)b, ᏶(a, z)b = pW 0 Y (a, z)b for a, b ∈ W 0. Then by (5-18), Ᏽ( · , z) and ᏶( · , z) are nonzero intertwining M0 W 0 operators of respective types W 0 W 0 and W 0 W 0 . By the fusion rules (5-17), M0 0 M0 the space IM0 W 0 W 0 of M -intertwining operators of type W 0 W 0 is of dimen- W 0 = 0 · 0 ⊂ 0 sion one. Likewise, dim IM0 W 0 W 0 1. Note that W M W and that Ᏽ(a, z)b + ᏶(a, z)b = Y˜ (a, z)b. M Let pM : U → M and pW : U → W be projections. Define Ᏺi ( · , z) and W Ᏺi ( · , z), i = 1, 2 by M W Ᏺi (a, z)w = pM Yi (a, z)w, Ᏺi (a, z)w = pW Yi (a, z)w

0 M W for a ∈ W and w ∈ W. Then Ᏺi ( · , z) and Ᏺi ( · , z) are intertwining opera- M W M + W = tors of type W 0 W and W 0 W , respectively. Clearly, Ᏺi (a, z)w Ᏺi (a, z)w M 0 τ 0 0 Yi (a, z)w. If Ᏺi ( · , z) = 0, then W ·W ⊂ W and so VL ·W = M ·W +W ·W ⊂ W. τ M This is a contradiction, since U is a simple VL -module. Hence Ᏺi ( · , z) 6= 0. Let W Ᏻi (a, z)v = Yi (a, z)v ∈ 0 ∈ W · W for a W , v M. Then Ᏻi ( , z) is a nonzero intertwining operator of type W 0 M 0 W W by (5-17). The space of M -intertwining operators IM0 W 0 M of type W 0 M is of M = W = dimension one by (5-17). Similarly, dim IM0 W 0 W dim IM0 W 0 W 1. There- M · = M · W · = W · W · = W · fore, Ᏺ2 ( , z) λᏲ1 ( , z), Ᏺ2 ( , z) µᏲ1 ( , z), and Ᏻ2 ( , z) γ Ᏻ1 ( , z) for some λ, µ, γ ∈ ރ with λ 6= 0 and γ 6= 0. Now,

M W W Yi (a, z1)Yi (b, z2)v = Ᏺi (a, z1) + Ᏺi (a, z1) Ᏻi (b, z2)v, M W W Yi (b, z2)Yi (a, z1)v = Ᏺi (b, z2) + Ᏺi (b, z2) Ᏻi (a, z1)v, ˜ W Yi (Y (a, z0)b, z2)v = Yi (Ᏽ(a, z0)b, z2)v + Ᏻi (᏶(a, z0)b, z2)v ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 505 for a, b ∈ W 0 and v ∈ M. Taking the image of both sides of the Jacobi identity − − −1 z1 z2 − −1 z2 z1 (5-26) z0 δ Yi (a, z1)Yi (b, z2)v z0 δ Yi (b, z2)Yi (a, z1)v z0 −z0 − = −1 z1 z0 ˜ z2 δ Yi (Y (a, z0)b, z2)v z2 under the projection pM , we obtain − − −1 z1 z2 M W − −1 z2 z1 M W (5-27) z0 δ Ᏺi (a, z1)Ᏻi (b, z2)v z0 δ Ᏺi (b, z2)Ᏻi (a, z1)v z0 −z0 − = −1 z1 z0 z2 δ Yi (Ᏽ(a, z0)b, z2)v. z2

Likewise, if we take the image of both sides of (5-26) under the projection pW , then − − −1 z1 z2 W W − −1 z2 z1 W W (5-28) z0 δ Ᏺi (a, z1)Ᏻi (b, z2)v z0 δ Ᏺi (b, z2)Ᏻi (a, z1)v z0 −z0 − = −1 z1 z0 W z2 δ Ᏻi (᏶(a, z0)b, z2)v. z2 Comparing Equation (5-28) for i = 1 and i = 2, we have z − z − −1 1 0 W = γ (µ 1)z2 δ Ᏻ1 (᏶(a, z0)b, z2)v 0, z2 M · = M · W · = W · W · = W · since Ᏺ2 ( , z) λᏲ1 ( , z), Ᏺ2 ( , z) µᏲ1 ( , z), and Ᏻ2 ( , z) γ Ᏻ1 ( , z). Now, z−1δ z1−z0 = z−1δ z2+z0 by [Frenkel et al. 1988, Proposition 8.8.5] and so 2 z2 1 z1 the above equation is equivalent to the following assertion.

k W γ (µ − 1)(z2 + z0) Ᏻ1 (᏶(a, z0)b, z2)v = 0 for all k ∈ ޚ. This implies that W γ (µ − 1)Ᏻ1 (᏶(a, z0)b, z2)v = 0, W ∈ [[ −1]] · W · since Ᏻ1 (᏶(a, z0)b, z2)v W((z0)) z2, z2 . Then since ᏶( , z) and Ᏻ1 ( , z) are nonzero, we conclude that µ = 1. 0 Next, we use Equation (5-27). Since Ᏽ(a, z0)b ∈ M ((z0)), we have

Y1(Ᏽ(a, z0)b, z2)v = Y2(Ᏽ(a, z0)b, z2)v by our assumption. Then it follows from (5-27) for i = 1, 2 that z − z − −1 1 0 = (λγ 1)z2 δ Y1(Ᏽ(a, z0)b, z2)v 0. z2 0 Since Ᏽ( · , z) 6= 0 and M is a simple (M , Y1)-module, a similar argument as above gives that λγ = 1. 506 KENICHIRO TANABE AND HIROMICHI YAMADA

For a ∈ M0, b ∈ W 0, v ∈ M, and w ∈ W,

W M W Yi (a+b, z)(v+w) = Yi (a, z)v+Yi (a, z)w+Ᏻi (b, z)v+ Ᏺi (b, z)+Ᏺi (b, z) w.

M W W Note that Yi (a, z)v, Ᏺi (b, z)w ∈ M((z)) and Yi (a, z)w, Ᏻi (b, z)v, Ᏺi (b, z)w ∈ W((z)). Deﬁne ϕ : U → U by ϕ(u) = λu if u ∈ M and ϕ(u) = u if u ∈ W. Since µ = 1 and λγ = 1, we can verify that Y2(a + b, z)ϕ(v + w) = ϕ Y1(a + b, z)(v + w) .

τ Thus ϕ is an isomorphism of VL -modules from (U, Y1) onto (U, Y2). This completes the proof. Remark 5.8. The proof of the above lemma is essentially the same as that of [Lam et al. 2005, Lemma C.3]. Consider the Jacobi identity for a, b ∈ W 0 and w ∈ W and take the images of both sides of the identity under the projections pM and pW , respectively. Then z − z z − z −1 1 2 M W − −1 2 1 M W z0 δ Ᏺi (a, z1)Ᏺi (b, z2)w z0 δ Ᏺi (b, z2)Ᏺi (a, z1)w z0 −z0 z − z = −1 1 0 M z2 δ Ᏺi (᏶(a, z0)b, z2)w, z2 z − z −1 1 2 W M + W W z0 δ Ᏻi (a, z1)Ᏺi (b, z2) Ᏺi (a, z1)Ᏺi (b, z2) w z0 z − z − −1 2 1 W M + W W z0 δ Ᏻi (b, z2)Ᏺi (a, z1) Ᏺi (b, z2)Ᏺi (a, z1) w −z0 z − z = −1 1 0 + W z2 δ Yi (Ᏽ(a, z0)b, z2) Ᏺi (᏶(a, z0)b, z2) w. z2 Each of these two equations gives the identical equations in case of i = 1 and i = 2 provided that µ = 1 and λγ = 1.

τ Theorem 5.9. There are exactly 30 inequivalent simple VL -modules. They are τ represented by the 30 simple VL -modules listed in Lemma 3.2. τ Proof. Let U be a simple VL -module. Then by Lemma 5.5, U contains a simple 0 τ M -submodule M isomorphic to a member of ᏹ. Since U is a simple VL -module, Lemma 5.6 implies that U = M⊕W for some simple M0-submodule W isomorphic to a member of ᐃ. In fact, the isomorphism class of W is uniquely determined by τ M. By Lemma 5.7, U admits a unique VL -module structure. Since ᏹ consists of τ 30 members, it follows that there are at most 30 inequivalent simple VL -modules. Hence the assertion holds. τ Theorem 5.10. VL is a rational vertex operator algebra. ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 507

τ Proof. It is sufﬁcient to show that every ގ-graded weak VL -module U is a sum of τ 0 0 simple VL -modules. Since M is rational, U is a direct sum of simple M -modules. L γ L λ Thus by Lemma 5.2, we may assume that U = γ ∈0 S ⊕ λ∈3 S , where Sγ is isomorphic to a member of ᏹ and Sλ is isomorphic to a member of ᐃ. We τ γ τ P τ γ know that VL · S is a simple VL -module by Lemma 5.6. Set N = γ ∈0 VL · S . Since U/N has no simple M0-submodule isomorphic to a member of ᏹ, it follows from Lemma 5.5 that U = N and the proof is complete. τ τ Corollary 5.11. The Zhu algebra A(VL ) of VL is a 51 dimensional semisimple associative algebra isomorphic to a direct sum of 23 copies of the one dimensional τ algebra ރ and 7 copies of the algebra Mat2(ރ) of 2×2 matrices. Moreover, A(VL ) is generated by [ω ˜ 1], [ω ˜ 2], [J], [K ], and [P]. τ τ Proof. Since VL is rational, A(VL ) is a ﬁnite dimensional semisimple associative algebra [Dong et al. 1998a, Theorem 8.1; Zhu 1996, Theorem 2.2.3]. We know τ 1 2 all the simple VL -modules and the action of [ω ˜ ], [ω ˜ ], [J], [K ], and [P] on their τ top levels in Section 4. Hence we can determine the structure of A(VL ) as in the assertion.

Appendix: Some fusion rules for M(0)

We give a proof of the fusion rules

i i W(0) × MT (τ )(ε) = WT (τ )(ε), i i i W(0) × WT (τ )(ε) = MT (τ )(ε) + WT (τ )(ε), i = 1, 2, ε = 0, 1, 2 of simple M(0)-modules in (3-25). τ ∼ 0 0 0 0 0 0 Recall that VL = M ⊕ W , where M = M(0) ⊗ Mt and W = W(0) ⊗ Wt . ˆ i i 0 ˆ i i 0 Set MT (τ )(ε) = MT (τ )(ε) ⊗ Mt and WT (τ )(ε) = WT (τ )(ε) ⊗ Wt , which are simple M0-modules. Then 0 ∼ ˆ ˆ VL (τ)(ε) = MT (τ)(ε) ⊕ WT (τ)(ε), 0 2 ∼ ˆ 2 ˆ 2 VL (τ )(ε) = MT (τ )(ε) ⊕ WT (τ )(ε)

0 as M -modules by (3-23) and (3-24). Denote by Y1( · , z) (resp. Y2( · , z)) the τ 0 0 2 vertex operator of the simple VL -module VL (τ)(ε) (resp. VL (τ )(ε)). Let pM : 0 ˆ 0 ˆ VL (τ)(ε)→ MT (τ)(ε) and pW : VL (τ)(ε)→ WT (τ)(ε) be projections. We also use 0 2 ˆ 2 the same symbol pM or pW to denote a projection from VL (τ )(ε) onto MT (τ )(ε) ˆ 2 or onto WT (τ )(ε). We ﬁx i = 1, 2 and ε = 0, 1, 2. For simplicity of notation, set ˆ ˆ i ˆ ˆ i M = MT (τ )(ε) and W = WT (τ )(ε). M W 0 Let Ᏺi (a, z)w = pM Yi (a, z)w and Ᏺi (a, z)w = pW Yi (a, z)w for a ∈ W and ˆ ∈ ˆ M · W · M w W. Then Ᏺi ( , z) and Ᏺi ( , z) are intertwining operators of type W 0 Wˆ and 508 KENICHIRO TANABE AND HIROMICHI YAMADA

ˆ W W = ∈ 0 ∈ ˆ W 0 Wˆ , respectively. Likewise, let Ᏻi (a, z)v Yi (a, z)v for a W and v M. ˆ W · W Then Ᏻi ( , z) is an intertwining operator of type W 0 Mˆ , since the fusion rule 0 0 0 0 0 ˆ ˆ 0 Wt × Mt = Wt of Mt -modules implies that W · M = span{an M | a ∈ W , n ∈ ޚ} ˆ W τ ˆ 0 0 ˆ ˆ is contained in W. If Ᏻi ( · , z) = 0, then VL · M = (M + W ) · M ⊂ M. This 0 0 2 τ is a contradiction, since VL (τ)(ε) and VL (τ )(ε) are simple VL -modules. Thus W M M τ ˆ ˆ Ᏻi ( · , z) 6= 0. Similarly, Ᏺi ( · , z) 6= 0. Indeed, if Ᏺi ( · , z) = 0, then VL ·W ⊂ W, W 0 ˆ ˆ which is a contradiction. Assume that Ᏺi ( · , z) = 0. Then W · W ⊂ M and so 0 0 ˆ ˆ 0 0 ˆ 0 0 ˆ τ ˆ W ·(W · W) ⊂ W. However, W ·(W · W) ⊃ (W · W )· W = VL · W by Lemma W 2.6 and (5-18). This contradiction implies that Ᏺi ( · , z) 6= 0. M W Restricting the three nonzero intertwining operators Ᏺi ( · , z), Ᏺi ( · , z), and W 0 0 Ᏻi ( · , z) to the ﬁrst component of each of the tensor products W = W(0) ⊗ Wt , ˆ i 0 ˆ i 0 M = MT (τ )(ε)⊗ Mt , and W = WT (τ )(ε)⊗ Wt , we obtain nonzero intertwining operators of type

i i i MT (τ )(ε) WT (τ )(ε) WT (τ )(ε) i , i i W(0) WT (τ )(ε) W(0) WT (τ )(ε) W(0) MT (τ )(ε) for M(0)-modules, respectively. 2 i i 3 Let N be one of MT (τ )(ε), WT (τ )(ε), i = 1, 2, ε = 0, 1, 2 and let N be any j j of the 20 simple M(0)-modules. Then the top level N(0) of N is of dimension one. By [Dong et al. 2004], the Zhu algebra A(M(0)) of M(0) is generated by [ω ˜ 1] [ ] ˜ 1 j and J . Moreover, we know the action of o(ω ) and o(J) on N(0). Thus, by an argument as in [Tanabe 2005, pp. 192–193], we can calculate that the dimension of

2 3 HomA(M(0))(A(W(0)) ⊗A(M(0)) N(0), N(0)) is at most one and it is equal to one if and only if the pair (N 2, N 3) is one of

i i i i i i (MT (τ )(ε), WT (τ )(ε)), (WT (τ )(ε), MT (τ )(ε)), (WT (τ )(ε), WT (τ )(ε))

0(0) for i = 1, 2, ε = 0, 1, 2. Note that W(0) was denoted by Wk in [Tanabe 2005]. Now, the desired fusion rules are obtained by [Li 1999a, Proposition 2.10 and Corollary 2.13].

References

[Abe and Dong 2004] T. Abe and C. Dong, “Classiﬁcation of irreducible modules for the ver- + tex operator algebra VL : general case”, J. Algebra 273:2 (2004), 657–685. MR 2005c:17040 Zbl 1051.17015 [Abe et al. 2004] T. Abe, G. Buhl, and C. Dong, “Rationality, regularity, and C2-coﬁniteness”, Trans. Amer. Math. Soc. 356:8 (2004), 3391–3402. MR 2005c:17041 Zbl 1070.17011 [Abe et al. 2005] T. Abe, C. Dong, and H. Li, “Fusion rules for the vertex operator algebra M(1) + and VL ”, Comm. Math. Phys. 253:1 (2005), 171–219. MR 2005i:17033 Zbl pre05076366 ORDER-THREE AUTOMORPHISMS ON A LATTICE VERTEX OPERATOR ALGEBRA 509

[Bouwknegt et al. 1996] P. Bouwknegt, J. McCarthy, and K. Pilch, The ᐃ3 algebra: Modules, semi-infinite cohomology and BV algebras, Lecture Notes in Physics M42, Springer, Berlin, 1996. MR 97m:17029 [Buhl 2002] G. Buhl, “A spanning set for VOA modules”, J. Algebra 254:1 (2002), 125–151. MR 2003m:17022 Zbl 1037.17032 [Dong 1993] C. Dong, “Vertex algebras associated with even lattices”, J. Algebra 161:1 (1993), 245–265. MR 94j:17023 Zbl 0807.17022 [Dong and Lepowsky 1996] C. Dong and J. Lepowsky, “The algebraic structure of relative twisted vertex operators”, J. Pure Appl. Algebra 110:3 (1996), 259–295. MR 98e:17036 Zbl 0862.17021 [Dong and Mason 1997] C. Dong and G. Mason, “On quantum Galois theory”, Duke Math. J. 86:2 (1997), 305–321. MR 97k:17042 Zbl 0890.17031 [Dong and Yamskulna 2002] C. Dong and G. Yamskulna, “Vertex operator algebras, generalized doubles and dual pairs”, Math. Z. 241:2 (2002), 397–423. MR 2003j:17038 Zbl 1073.17009 [Dong et al. 1997] C. Dong, H. Li, and G. Mason, “Regularity of rational vertex operator algebras”, Adv. Math. 132:1 (1997), 148–166. MR 98m:17037 Zbl 0902.17014 [Dong et al. 1998a] C. Dong, H. Li, and G. Mason, “Twisted representations of vertex operator algebras”, Math. Ann. 310:3 (1998), 571–600. MR 99d:17030 Zbl 0890.17029 [Dong et al. 1998b] C. Dong, H. Li, G. Mason, and S. P. Norton, “Associative subalgebras of the Griess algebra and related topics”, pp. 27–42 in The Monster and Lie algebras (Columbus, OH, 1996), edited by J. Ferrar and K. Harada, Ohio State Univ. Math. Res. Inst. Publ. 7, de Gruyter, Berlin, 1998. MR 99k:17048 Zbl 0946.17011 [Dong et al. 2000] C. Dong, H. Li, and G. Mason, “Modular-invariance of trace functions in orbifold theory and generalized Moonshine”, Comm. Math. Phys. 214:1 (2000), 1–56. MR 2001k:17043 Zbl 1061.17025 [Dong et al. 2004] C. Dong, C. H. Lam, K. Tanabe, H. Yamada, and K. Yokoyama, “ޚ3 symmetry and W3 algebra in lattice vertex operator algebras”, Pacific J. Math. 215:2 (2004), 245–296. MR 2005c:17045 Zbl 1055.17013 [Frenkel et al. 1988] I. Frenkel, J. Lepowsky, and A. Meurman, Vertex operator algebras and the Monster, Pure and Applied Mathematics 134, Academic Press, Boston, 1988. MR 90h:17026 Zbl 0674.17001 [Frenkel et al. 1993] I. B. Frenkel, Y.-Z. Huang, and J. Lepowsky, On axiomatic approaches to vertex operator algebras and modules, Mem. Amer. Math. Soc. 494, Amer. Math. Soc., Providence, RI, 1993. MR 94a:17007 Zbl 0789.17022 [Gaberdiel and Neitzke 2003] M. R. Gaberdiel and A. Neitzke, “Rationality, quasirationality and finite W-algebras”, Comm. Math. Phys. 238:1-2 (2003), 305–331. MR 2004g:81084 Zbl 1042.17025 [Kitazume et al. 2000a] M. Kitazume, C. H. Lam, and H. Yamada, “Decomposition of the Moon- shine vertex operator algebra as Virasoro modules”, J. Algebra 226:2 (2000), 893–919. MR 2001f: 17055 Zbl 0986.17010 [Kitazume et al. 2000b] M. Kitazume, M. Miyamoto, and H. Yamada, “Ternary codes and vertex operator algebras”, J. Algebra 223:2 (2000), 379–395. MR 2000m:17032 Zbl 0977.17026 [Kitazume et al. 2003] M. Kitazume, C. H. Lam, and H. Yamada, “3-state Potts model, Moonshine vertex operator algebra, and 3A-elements of the Monster group”, Int. Math. Res. Not. 2003:23 (2003), 1269–1303. MR 2004b:17051 Zbl 1037.17034 [Lam and Yamada 2000] C. H. Lam and H. Yamada, “Z2 × Z2 codes and vertex operator algebras”, J. Algebra 224:2 (2000), 268–291. MR 2001d:17029 Zbl 1013.17025 [Lam et al. 2005] C. H. Lam, H. Yamada, and H. Yamauchi, “McKay’s observation and vertex operator algebras generated by two conformal vectors of central charge 1/2”, IMRP Int. Math. Res. Pap. 2005:3 (2005), 117–181. MR 2006h:17034 Zbl 1082.17015 510 KENICHIRO TANABE AND HIROMICHI YAMADA

[Lepowsky 1985] J. Lepowsky, “Calculus of twisted vertex operators”, Proc. Nat. Acad. Sci. U.S.A. 82:24 (1985), 8295–8299. MR 88f:17030 Zbl 0579.17010 [Lepowsky and Li 2004] J. Lepowsky and H. Li, Introduction to vertex operator algebras and their representations, Progress in Mathematics 227, Birkhäuser, Boston, 2004. MR 2004k:17050 Zbl 1055.17001 [Li 1999a] H. Li, “Determining fusion rules by A(V )-modules and bimodules”, J. Algebra 212:2 (1999), 515–556. MR 2000d:17032 Zbl 0977.17027 [Li 1999b] H. Li, “Some finiteness properties of regular vertex operator algebras”, J. Algebra 212:2 (1999), 495–514. MR 2000c:17044 Zbl 0953.17017 [Li 2001] H. Li, “The regular representation, Zhu’s A(V )-theory, and induced modules”, J. Algebra 238:1 (2001), 159–193. MR 2003f:17039 Zbl 1060.17015 [Miyamoto 2001] M. Miyamoto, “3-state Potts model and automorphisms of vertex operator algebras of order 3”, J. Algebra 239:1 (2001), 56–76. MR 2002c:17042 Zbl 1022.17020 [Miyamoto and Tanabe 2004] M. Miyamoto and K. Tanabe, “Uniform product of Ag,n(V ) for an orbifold model V and G-twisted Zhu algebra”, J. Algebra 274:1 (2004), 80–96. MR 2005d:17037 Zbl 1046.17009 [Nagatomo and Tsuchiya 2005] K. Nagatomo and A. Tsuchiya, “Conformal field theories associated to regular chiral vertex operator algebras. I. Theories over the projective line”, Duke Math. J. 128:3 (2005), 393–471. MR 2006e:81258 Zbl 1074.81065 [Tanabe 2005] K. Tanabe, “On intertwining operators and finite automorphism groups of vertex operator algebras”, J. Algebra 287:1 (2005), 174–198. MR 2006e:17041 Zbl 02173135 [Wang 1993] W. Wang, “Rationality of Virasoro vertex operator algebras”, Internat. Math. Res. Notices 1993:7 (1993), 197–211. MR 94i:17034 Zbl 0791.17029 [Yamauchi 2004] H. Yamauchi, “Modularity on vertex operator algebras arising from semisimple primary vectors”, Internat. J. Math. 15:1 (2004), 87–109. MR 2005i:17036 Zbl 1052.17018 + [Yamskulna 2004] G. Yamskulna, “C2-cofiniteness of the vertex operator algebra VL when L is a rank one lattice”, Comm. Algebra 32:3 (2004), 927–954. MR 2005h:17053 Zbl 1093.17008 [Zhu 1996] Y. Zhu, “Modular invariance of characters of vertex operator algebras”, J. Amer. Math. Soc. 9:1 (1996), 237–302. MR 96c:17042 Zbl 0854.17034

Received August 11, 2005. Revised January 11, 2006.

KENICHIRO TANABE DEPARTMENT OF MATHEMATICS HOKKAIDO UNIVERSITY KITA 10, NISHI 8, KITA-KU SAPPORO,HOKKAIDO 060-0810 JAPAN [email protected]

HIROMICHI YAMADA DEPARTMENT OF MATHEMATICS HITOTSUBASHI UNIVERSITY NAKA 2-1, KUNITACHI TOKYO 186-8601 JAPAN [email protected] CONTENTS

Volume 230, no. 1 and no. 2

Federico Ardila: Computing the Tutte polynomial of a hyperplane arragement 1 Elliot Benjamin, Franz Lemmermeyer and Chip Snyder: On the unit group of some multiquadratic number ﬁelds 27 Pierre Bieliavsky, Philippe Bonneau and Yoshiaki Maeda: Universal deformation formulae, symplectic Lie groups and symmetric spaces 41 Martin Bohner and Christopher C. Tisdell: Oscillation and nonoscillation of forced second order dynamic equations 59 Philippe Bonneau with Pierre Bieliavsky and Yoshiaki Maeda 41 John Brevik: Curves on normal rational cubic surfaces 73 Silvano Delladio: A result about C3-rectiﬁability of Lipschitz curves 257 Stefan Friedl: Reidemeister torsion, the Thurston norm and Harvey’s invariants 271 Patrick M. Gilmer: Arf invariants of real algebraic curves 297 Peter Hästö: Isometries of the quasihyperbolic metric 315 Naihong Hu and Qian Shi: The two-parameter quantum group of exceptional type G2 and Lusztig’s symmetries 327 Takeshi Katsura: Ideal structure of C∗-algebras associated with C∗-correspondences 107 Maxim Kazarian with Boris Shapiro 233 Philippe Laurençot: Convergence to steady states for a one-dimensional viscous Hamilton–Jacobi equation with Dirichlet boundary conditions 347 Franz Lemmermeyer with Elliot Benjamin and Chip Snyder 27 Yoshiaki Maeda with Pierre Bieliavsky and Philippe Bonneau 41 Laurent Mazet: Uniqueness results for constant mean curvature graphs 365 Michael P. McCooey: Groups that act pseudofreely on S2 × S2 381 John Douglas Moore: Nondegeneracy of coverings of minimal tori and Klein bottles in Riemannian manifolds 147 Frank Müller and Sven Winklmann: Projectability and uniqueness of F-stable immersions with partially free boundaries 409 400

Akito Nomura: Unramified 3-extensions over cyclic cubic fields 167 Roger D. Patterson, Alfred J. van der Poorten and Hugh C. Williams: Characterization of a generalized Shanks sequence 185 Hans Schoutens: Asymptotic homological conjectures in mixed characteristic 427 Kenneth J. Shackleton: Combinatorial rigidity in curve complexes and mapping class groups 217 Boris Shapiro and Maxim Kazarian: A Giambelli-type formula for subbundles of the tangent bundle 233 Qian Shi with Naihong Hu 327 Chip Snyder with Elliot Benjamin and Franz Lemmermeyer 27 Kenichiro Tanabe and Hiromichi Yamada: The fixed point subalgebra of a lattice vertex operator algebra by an automorphism of order three 469 Christopher C. Tisdell with Martin Bohner 59 Hugh C. Williams with Roger D. Patterson and Alfred J. van der Poorten 185 Sven Winklmann with Frank Müller 409 Hiromichi Yamada with Kenichiro Tanabe 469 Alfred J. van der Poorten with Roger D. Patterson and Hugh C. Williams 185 Guidelines for Authors

Authors may submit manuscripts at pjm.math.berkeley.edu/about/journal/submissions.html and choose an editor at that time. Exceptionally, a paper may be submitted in hard copy to one of the editors; authors should keep a copy. By submitting a manuscript you assert that it is original and is not under consideration for publication elsewhere. Instructions on manuscript preparation are provided below. For further information, visit the web address above or write to pacifi[email protected] or to Pacific Journal of Mathematics, University of California, Los Angeles, CA 90095–1555. Correspondence by email is requested for convenience and speed. Manuscripts must be in English, French or German. A brief abstract of about 150 words or less in English must be included. The abstract should be self-contained and not make any reference to the bibliography. Also required are keywords and subject classification for the article, and, for each author, postal address, affiliation (if appropriate) and email address if available. A home-page URL is optional.

Authors are encouraged to use LATEX, but papers in other varieties of TEX, and exceptionally in other formats, are acceptable. At submission time only a PDF file is required; follow the instructions at the web address above. Carefully preserve all relevant files, such as LATEX sources and individual files for each figure; you will be asked to submit them upon acceptance of the paper. Bibliographical references should be listed alphabetically at the end of the paper. All references in the bibliography should be cited in the text. Use of BibTEX is preferred but not required. Any bibliographical citation style may be used but tags will be converted to the house format (see a current issue for examples). Figures, whether prepared electronically or hand-drawn, must be of publication quality. Figures prepared electronically should be submitted in Encapsulated PostScript (EPS) or in a form that can be converted to EPS, such as GnuPlot, Maple or Mathematica. Many drawing tools such as Adobe Illustrator and Aldus FreeHand can produce EPS output. Figures containing bitmaps should be generated at the highest possible resolution. If there is doubt whether a particular figure is in an acceptable format, the authors should check with production by sending an email to pacifi[email protected]. Each figure should be captioned and numbered, so that it can float. Small figures occupying no more than three lines of vertical space can be kept in the text (“the curve looks like this:”). It is acceptable to submit a manuscript will all figures at the end, if their placement is specified in the text by means of comments such as “Place Figure 1 here”. The same considerations apply to tables, which should be used sparingly. Forced line breaks or page breaks should not be inserted in the document. There is no point in your trying to optimize line and page breaks in the original manuscript. The manuscript will be reformatted to use the journal’s preferred fonts and layout. Page proofs will be made available to authors (or to the designated corresponding author) at a Web site in PDF format. Failure to acknowledge the receipt of proofs or to return corrections within the requested deadline may cause publication to be postponed. PACIFIC JOURNAL OF MATHEMATICS Mathematics of Journal Pacific

Volume 230 No. 2 April 2007 Paciﬁc

Computing the Tutte polynomial of a hyperplane arragement 1 Journal of FEDERICO ARDILA On the unit group of some multiquadratic number ﬁelds 27 ELLIOT BENJAMIN,FRANZ LEMMERMEYERAND CHIP SNYDER Universal deformation formulae, symplectic Lie groups and symmetric spaces 41 Mathematics PIERRE BIELIAVSKY,PHILIPPE BONNEAUAND YOSHIAKI MAEDA Oscillation and nonoscillation of forced second order dynamic equations 59 MARTIN BOHNERAND CHRISTOPHER C.TISDELL Curves on normal rational cubic surfaces 73 JOHN BREVIK

Ideal structure of C∗-algebras associated with C∗-correspondences 107 2007 TAKESHI KATSURA Nondegeneracy of coverings of minimal tori and Klein bottles in Riemannian

manifolds 147 2 No. 230, Vol. JOHN DOUGLAS MOORE Unramified 3-extensions over cyclic cubic fields 167 AKITO NOMURA Characterization of a generalized Shanks sequence 185 ROGER D.PATTERSON,ALFRED J. VAN DER POORTEN AND HUGH C.WILLIAMS Combinatorial rigidity in curve complexes and mapping class groups 217 KENNETH J.SHACKLETON A Giambelli-type formula for subbundles of the tangent bundle 233 BORIS SHAPIROAND MAXIM KAZARIAN A result about C3-rectifiability of Lipschitz curves 257 SILVANO DELLADIO Reidemeister torsion, the Thurston norm and Harvey’s invariants 271 STEFAN FRIEDL Volume 230 No. 2 April 2007 Arf invariants of real algebraic curves 297 PATRICK M.GILMER Isometries of the quasihyperbolic metric 315 PETER HÄSTÖ

The two-parameter quantum group of exceptional type G2 and Lusztig’s symmetries 327 NAIHONG HUAND QIAN SHI Convergence to steady states for a one-dimensional viscous Hamilton–Jacobi equation with Dirichlet boundary conditions 347 PHILIPPE LAURENÇOT Uniqueness results for constant mean curvature graphs 365 LAURENT MAZET Groups that act pseudofreely on S2 × S2 381 MICHAEL P. MCCOOEY Projectability and uniqueness of F-stable immersions with partially free boundaries 409 FRANK MÜLLERAND SVEN WINKLMANN Asymptotic homological conjectures in mixed characteristic 427 HANS SCHOUTENS The ﬁxed point subalgebra of a lattice vertex operator algebra by an automorphism of order three 469 KENICHIRO TANABEAND HIROMICHI YAMADA