PROCEEDINGS OF THE AMERICAN MATHEMATICAL SOCIETY Volume 142, Number 4, April 2014, Pages 1157–1163 S 0002-9939(2014)12033-8 Article electronically published on January 30, 2014

AUTOMORPHISMS OF AFFINE SMOOTH VARIETIES

ZBIGNIEW JELONEK AND TOMASZ LENARCIK

(Communicated by Harm Derksen)

Abstract. Let k be an algebraically closed field. If X is a smooth over k and H ⊂ X is a very ample without ruled com- ponents, then the group Aut(X \ H) is finite and equal to StabX (H)={φ ∈ Aut(X):φ(H)=H}.

1. Introduction Let k be an algebraically closed field. Let us recall that an irreducible variety V is ruled if it is birationally equivalent to a cylinder W × P1. The aim of this note is to prove the following:

Theorem 1. Let X be a smooth projective irreducible variety of dimension > 1. Let H be a hypersurface in X which does not have ruled components. If H is a very ample divisor, then the group Aut(X \ H) is finite. Moreover, Aut(X \ H)= StabX (H). This theorem for k = C was proved (in a slightly weaker version) by the first author (see [7]). We apply this theorem to study Zariski open affine subsets of a smooth complete intersection in Pn. In particular, we get in this way the following results:

Corollary 1. Let X be a smooth complete intersection of multi-degree d1,...,dr Pn − ≥ ≥ r in where n 2 r 0 and set d =Σi=1di.Lets be a natural number and − Yi,i=1,...,s, a collection of smooth of degree at least n +1 d. ∩ s If each Yi meets X strictly transversely and V = X ( i=1 Yi), then the group Aut(X \ V ) is finite and is equal to StabX (V ). If k = C we can use some results of Pukhlikov and Kollar to obtain the following improvement of Corollary 43 in [8]:

Corollary 2. Let k = C. Let n ≥ 4 and V be a hypersurface in Pn such that every irreducible component H of V is a smooth variety of degree ≥ n. Then every poly- n nomial automorphism of the variety P \V can be extended to a linear automorphism n n of P and the group Aut(P \ V ) is finite. Moreover, for V = Hi with sufficiently

Received by the editors November 2, 2011 and, in revised form, May 11, 2012. 2010 Mathematics Subject Classification. Primary 13C10, 14J70, 14R99. Key words and phrases. Algebraic automorphism, complete intersection, affine variety. The first author acknowledges support of the Polish Ministry of Science and Higher Education, grant MNiSW N N201 420939, 2010–2013.

c 2014 American Mathematical Society Reverts to public domain 28 years from publication 1157

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 1158 ZBIGNIEW JELONEK AND TOMASZ LENARCIK

general irreducible hypersurfaces Hi, the extendability and finiteness assertions hold whenever deg Hi ≥ 2[(n +2)/3] ([x] denotes the smallest integer ≥ x). In the proof we use the methods from [7] and [12] with some necessary modifi- cations.

2. Terminology For an X we denote by Aut(X) the set of all regular automor- phisms of X.IfV ⊂ X is a subvariety of X,thenwedenotebyAutX (V )the subgroup of Aut(V ) consisting of elements induced by automorphisms of X leaving V invariant, and by StabX (V ) the subgroup of Aut(X) consisting of the elements which leave V invariant. Let f : X −→Y be a birational map and assume that Y is normal. The closed subset E(f)={x ∈ X|f is not a local isomorphism at x} is called the exceptional set of f. Let X be a smooth projective irreducible variety. A canonical divisor on X will be denoted by K(X). Let D ∈ Div(X) be a divisor. The linear system associated with D, considered as a , will be denoted by L(D). If X ⊂ Pn, then by LX (d) we mean the linear system which is cut out on X by hypersurfaces of degree d. If H is a very ample divisor on X, then by a canonical embedding given by n H we mean the embedding iH : X −→ P determined by a certain basis of the n space Γ(X, O(H)). In particular, if iH : X −→ P is a canonical embedding, then iH (X) is not contained in any hyperplane and the linear system LX (1) = L(H)is complete. If Vis a hypersurface in X we also consider V to be a reduced divisor, namely, r V := i=1 Vi,whereVi, i =1,...,r, are components of V ; conversely, any reduced divisor we will consider as a hypersurface. The statement “a very general member of a variety X has the property W” ⊂ means that there exist countably many proper and closed subsets Fi X such \ ∞ that every element of X i=1 Fi has the property W. 3. Tools The main tools of our proof are the following classical results: Theorem 3.1 (Abhyankar). Let X and Y be normal varieties and f : X −→ Y a birational morphism of finite type. If Y is smooth, then every exceptional divisor of f is ruled. Proof. See [6] or [1].  Theorem 3.2 (Rosenlicht). Suppose that an infinite affine group G acts faithfully on an algebraic (irreducible) variety X.ThenX is ruled. Proof. See [2] or [11] for the original statement.  The following theorem can be deduced, e.g., from Theorem 10.9 in [3]: Theorem 3.3. Let G be an infinite affine group. Then G contains either the ∗ additive group Ga =(k, +, 0) or the multiplicative group Gm =(k , ·, 1).

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use AUTOMORPHISMS OF AFFINE SMOOTH VARIETIES 1159

We also need the following version of Zariski’s Main Theorem (see [6]): Theorem 3.4 (Zariski). Let Y be a normal factorial variety, X a normal variety and f : X −→ Y a birational morphism. Then every irreducible component of the exceptional set E of f is of codimension one in X. Proof. See [6, Theorem 1.5, p. 289]. 

4. Main result We now prove our main result. Lemma 4.1 (see [7], [12]). Let X, Y be projective smooth irreducible varieties and H, H be hypersurfaces in X and Y respectively which do not have ruled components. If φ : X \ H −→ Y \ H is a regular isomorphism, then there exist closed subsets S, S of X, Y respectively which have codimension ≥ 2 such that φ extends to an isomorphism Φ:X \ S −→ Y \ S.Inotherwords,φ extends to an isomorphism in codimension 1. Proof. Let S denote the set of indeterminacy of ϕ.SinceX is smooth and projec- tive, codim S ≥ 2 (see [5, Thm. 2.17]). So we have a well defined morphism ϕ : X \ S =: U −→ Y. By Theorem 3.4, the irreducible components of the exceptional set, which is clearly contained in H, are divisors. From Theorem 3.1 it follows that every exceptional divisor is necessarily ruled. Since, by assumption, the components of H are not ruled, we have that the mapping ϕ has no exceptional locus. Now by Theorem 3.4 it is an isomorphism onto some open subset V ⊂ Y containing Y \H. By symmetry the conclusion follows.  We also need the following important result (see [7] and [12]): Lemma 4.2. Let X be a smooth projective irreducible variety. Let H be a hypersur- face in X which does not have ruled components. If H is an ample divisor, then the group Aut(X \ H) is a linear algebraic group. Moreover, Aut(X \ H)=StabX (H). Proof. We can assume that H is a very ample divisor (if not, we consider mH for m  0). Let φ ∈ Aut(X \ H). By Lemma 4.1, there exist open subsets U, V of X such that codim X \U ≥ 2, codim X \V ≥ 2 and we can extend φ to an isomorphism φ : U −→ V. By φ(X \ H) ⊂ X \ H we get (φ)−1(H ∩ V )=H ∩ U. This yields ∗ (1) φ L(H)|V = L(H)|U .

Since by assumption L(H) is very ample, there exists a closed embedding iH : n X −→ P given by sections s0,s1,...,sn which forms a basis of Γ(X, L(H)). Recall that codim X \ U ≥ 2 and codim X \ V ≥ 2. It follows that the restriction maps give rise to isomorphisms

(2) Γ(V,L(H)|V )  Γ(X, L(H))  Γ(U, L(H)|U ).

In particular the sequences {si|U }, {si|V } form bases of the corresponding linear ∗ spaces. By (1), it follows that each section si|V ◦φ ∈ φ L(H)|U can be expressed as

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 1160 ZBIGNIEW JELONEK AND TOMASZ LENARCIK

a linear combination of s0|U ,s1|U ,...,sn|U . In other words, there exists a (linear) n n automorphism Q : P −→ P extending φ, or more precisely Q ◦ i|U = i|V ◦ φ. Clearly Q|X ∈ StabX (H). 

Remark 4.3. The proof shows that if m is chosen so that mH is very ample, then the embedding i : X ⊂ Pn given by mH has the property that every automorphism of X \ H extends to an automorphism of Pn. More precisely, there is a hyperplane π ⊂ Pn such that H = X ∩ π and Aut(X \ H)={g ∈ PGL(n):g(X)=X, g(H)=H}. Remark 4.4. In Lemma 4.2 we can replace the assumption that H is an ample divisor by the weaker assumption that for some m ∈ Z the divisor mK(X)+H is ample. To prove our main result we need additional technical lemmas: Lemma 4.5. Let H be a subgroup of the group of affine (linear) automorphisms of kn, consisting of mappings of the form T : x → αx + β with α ∈ k∗ and β ∈ kn. Then each one dimensional subgroup G of H is either of the form k∗ t −→ tx + ta − a for some a ∈ kn or (in suitable coordinates) of the form k+ t −→ x +(t, 0,...,0). ∼ ∼ ∗ Proof. First assume that G = Gm. For every t ∈ Gm = k the element gt ∈Hhas aform

gt = α(t)x +(β1(t),...,βn(t)), −1 where α, βj ∈ k[t, t ]. Since the mapping t → gt is a homomorphism, we get α(t)α(t)=α(tt). This easily implies that α(t)=ts for some s ∈ Z. Moreover, we   s   have α(t)βi(t )+βi(t)=βi(tt ), i.e., t βi(t )+βi(t)=βi(tt ). This is only possible s if β(t)=ait − ai for some ai ∈ k. Since the mapping t → gt is an embedding, we have s ∈{−1, 1}. Now it is enough to take a =(a1,...,an) and if necessary change t−1 by t. ∼ ∼ Now assume that G = Ga. Again for every t ∈ Ga = k the element gt ∈Hhas the form

gt = α(t)x +(β1(t),...,βn(t)),  where α, βj ∈ k[t]. Since the mapping t → gt is a homomorphism, we get α(t)α(t )=   α(t + t ). This easily implies that α(t)=1. Moreover, we have α(t)βi(t )+βi(t)=    βi(t + t ), i.e., βi(t )+βi(t)=βi(t + t ). In particular βi is an additive polynomial and gt = x +(β1(t),...,βn(t)). Hence, if char k =0,thenβi(t)=cit.Ifchark =  j ni p p>0, then βi(t)= i=o cijt . Moreover, by assumption, the mapping β : k n t → (β1(t),...,βn(t)) ∈ k is an embedding (and an additive homomorphism). If char k =0,thenβ(k) is a line in kn and there is a linear transformation T : kn −→ kn such that T ◦ β(t)=(t, 0,...,0). If char k = p>0 we show that there is a polynomial additive isomorphism T : kn → kn such that T ◦ β(t)=(t, 0,...,0). We prove this by induction on d =maxdegβi. If d =1,thenT is a linear isomorphism. s ps Otherwise, let d = p =degβi > 1 be maximal. Hence βi = cit + ... lower terms. pr Since β is an embedding, there exists j = i such that βj = cj t + ... lower terms with cj = 0. Since the permutation of variables is an additive isomorphism, we can

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use AUTOMORPHISMS OF AFFINE SMOOTH VARIETIES 1161

assume that i>j.Consider the following additive isomorphism: − − −1 ps r T (x1,...,xn)=(x1,x2,...,xj cicj xi ,xj+1,...,xn). ◦   s  We have T β = β , where deg βi

1. Let H be a hypersurface in X which does not have uniruled components. If H is a very ample divisor, then the group Aut(X \ H) is finite. Moreover, Aut(X \ H)= StabX (H).

Proof. Since we have proved that StabX (H) → Aut(X \ H)isanisomorphism,we only need to verify that StabX (H) is finite. From now on, we consider X to be a n closed subvariety of P such that O(1)|X = L(H). In particular the closed subset H ⊂ X can be cut out from X by some hyperplane π ⊂ Pn, and we can identify n the group StabX (H) with a subgroup of Aut(P ) (see Remark 4.3). By Lemma 4.7, H is not contained in any proper linear subspace of π. Suppose that the group StabX (H) is not finite. Since it is an affine group, we have by Theorem 3.3 that StabX (H) contains a one dimensional subgroup G isomorphic either to Ga or Gm. By assumption G acts on H in a trivial way, so in particular, by Lemma 4.7 every element of G is trivial on the hyperplane π. Choose in Pn an affine system of coordinates such that the hyperplane π is the hyperplane at infinity. In particular G ⊂H,whereH is the group as in Lemma 4.5. By Lemma 4.5 we have two possibilities: ∗ n (3) G = {gt : x −→ tx + ta − a, t∈ k ,a∈ k } or (after change of coordinates in kn)

(4) G = {gt : x −→ x +(t, 0,...,0) ,t∈ k}. In case (3) the group G has a fixed point P = a. Since every line through P is stable under G,wehavethatX is a cone with vertex P. But the only nonsingular cones are linear subspaces of Pn.SinceX is not contained in any hyperplane we have X = Pn. This implies H = π. Since π is a ruled variety it is a contradiction. Hence we can assume case (4). Let X = X \ H ⊂ kn. Choose a point a =    (a1,...,an) ∈ X and let V = {x ∈ X : x1 = a1}. It is easy to see that X =  V × G = V × k. But now, there is an action of the group Gm on X ,andwego back to case (3). So it is a contradiction again. 

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 1162 ZBIGNIEW JELONEK AND TOMASZ LENARCIK

5. Application Here we give some simple applications of our main result. We will start with the following result (see [6, Corollary 1.11, p. 189]): 0 m Lemma 5.1. Let X be a smooth, proper and ruled variety. Then H (X, KX )=0 for every m>0. Now we can use Lemma 4.2 to study open affine subsets of a complete intersection in Pn:

Corollary 5.2. Let X be a smooth complete intersection of multi-degree d1,...,dr Pn − ≥ ≥ r in where n 2 r 0 and set d =Σi=1di.Lets be a natural number and − Yi,i=1,...,s, be a collection of smooth hypersurfaces of degree at least n +1 d. ∩ s If each Yi meets X strictly transversely and V = X ( i=1 Yi), then the group Aut(X \ V ) is finite and it is equal to StabX (V ). O − − r Proof. By the we have K(X)= (d n 1), where d =Σi=1di. Let ai =deg Yi and Vi = X ∩ Yi. Of course Vi are very ample divisors on X. Thus K(Vi)=O(d − n − 1+ai), and since d − n − 1+ai ≥ 0, we have that the canonical sheaf K(Vi) has a section. By Lemma 5.1, this implies that Vi is not s ruled. Moreover, V = i=1 Vi is a very ample divisor, because the union of very ample divisors is again a very ample divisor (see [5, 7.18, p. 252]).  In particular we have: Corollary 5.3. Let n ≥ 2 and V be a hypersurface in Pn such that every irreducible component of V is a smooth variety of degree ≥ n +1. Then every polynomial automorphism of the variety Pn \ V can be extended to a linear automorphism of Pn and the group Aut(Pn \ V ) is finite. For k = C Pukhlikov proved (see [9], [10]) that for n ≥ 4asmoothhypersurface V ⊂ Pn(C)ofdegreed ≥ n is birationally superrigid and in particular it cannot be fibered into uniruled varieties. This implies that V is not ruled. Moreover Kollar ([6, Theorem 5.14, p. 280]) proved that a very general hypersurface V ⊂ Pn(C)of degree d ≥ 2[(n+2)/3] is not ruled. Putting these together we see that Corollary 5.3 for k = C can be improved in the following way: Corollary 5.4. Let k = C. Let n ≥ 4 and V be a hypersurface in Pn such that every irreducible component H of V is a smooth variety of degree ≥ n.Then every polynomial automorphism of the variety Pn \ V can be extended to a linear n n automorphism of P and the group Aut(P \ V ) is finite. Moreover, for V = Hi with sufficiently general irreducible hypersurfaces Hi, the extendability and finiteness assertion hold whenever deg Hi ≥ 2[(n +2)/3] ([x] denotes the smallest integer ≥ x). References

[1] Shreeram Abhyankar, On the valuations centered in a local domain,Amer.J.Math.78 (1956), 321–348. MR0082477 (18,556b) [2] A. Bialynicki-Birula, On fixed point schemes of actions of multiplicative and additive groups, Topology 12 (1973), 99–103. MR0313261 (47 #1816) [3] Armand Borel, Linear algebraic groups, Notes taken by Hyman Bass, W. A. Benjamin, Inc., New York-Amsterdam, 1969. MR0251042 (40 #4273) [4] Robin Hartshorne, , Graduate Texts in Mathematics, No. 52, Springer- Verlag, New York, 1977. MR0463157 (57 #3116)

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use AUTOMORPHISMS OF AFFINE SMOOTH VARIETIES 1163

[5] Shigeru Iitaka, Algebraic geometry. An introduction to of algebraic va- rieties, North-Holland Mathematical Library, 24, Graduate Texts in Mathematics, vol. 76, Springer-Verlag, New York, 1982. MR637060 (84j:14001) [6] J´anos Koll´ar, Rational curves on algebraic varieties, Ergebnisse der Mathematik und ihrer Grenzgebiete. 3. Folge. A Series of Modern Surveys in Mathematics [Results in Mathematics and Related Areas. 3rd Series. A Series of Modern Surveys in Mathematics], vol. 32, Springer- Verlag, Berlin, 1996. MR1440180 (98c:14001) [7] Zbigniew Jelonek, Affine smooth varieties with finite group of automorphisms,Math.Z.216 (1994), no. 4, 575–591, DOI 10.1007/BF02572340. MR1288046 (95d:14046) [8] Zbigniew Jelonek, Irreducible identity sets for polynomial automorphisms,Math.Z.212 (1993), no. 4, 601–617, DOI 10.1007/BF02571679. MR1214049 (94c:14014) [9] A. V. Pukhlikov, Birationally rigid Fano hypersurfaces (Russian, with Russian summary), Izv. Ross. Akad. Nauk Ser. Mat. 66 (2002), no. 6, 159–186, DOI 10.1070/IM2002v066n06ABEH000413; English transl., Izv. Math. 66 (2002), no. 6, 1243– 1269. MR1970356 (2004a:14012b) [10] Aleksandr V. Pukhlikov, Birational automorphisms of higher-dimensional algebraic varieties, Proceedings of the International Congress of Mathematicians, Vol. II (Berlin, 1998), Doc. Math., 1998, Extra Vol. II, pp. 97–107 (electronic). MR1648060 (99k:14026) [11] Maxwell Rosenlicht, Some basic theorems on algebraic groups,Amer.J.Math.78 (1956), 401–443. MR0082183 (18,514a) [12] Burt Totaro, The automorphism group of an affine quadric, Math. Proc. Cambridge Philos. Soc. 143 (2007), no. 1, 1–8, DOI 10.1017/S0305004107000357. MR2340971 (2008j:11050)

Instytut Matematyczny PAN, ul. Sniadeckich´ 8, 00-956 Warszawa, Poland E-mail address: [email protected] Faculty of Mathematics and Computer Science, Jagiellonian Univeristy, ul. prof. StanislawaLojasiewicza 6, 30-348 Krakow,´ Poland E-mail address: [email protected]

License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use