LATTICE SIZE OF PLANE CONVEX BODIES

ANTHONY HARRISON, JENYA SOPRUNOVA, AND PATRICK TIERNEY

Abstract. The lattice size ls∆(P ) of a lattice polygon P with respect to the standard simplex ∆ was introduced and studied by Castryck and Cools in the context of simplification of the defining equation of an algebraic curve. Earlier, Schicho provided an “onion skins” algorithm for mapping a lattice polygon P into a small integer multiple of the standard simplex, based on passing successively to the convex hull of the interior lattice points of P . Castryck and Cools showed that this algorithm computes the lattice size of P . In this paper we show that for a plane convex body P a reduced basis of Z2 computes the lattice size. This provides a lattice reduction algorithm for comput- ing the lattice size, which works for any convex body P ⊂ R2 and outperforms the “onion skins” algorithm in the case when P is a lattice polygon.

Introduction The central object of this paper is the lattice size of P , where P is a plane convex body. In the case when P is a lattice polygon, this invariant was defined in [5] in the context of simplification of the defining equation of an algebraic curve. This notion appeared implicitly earlier in the work of Arnold [1], B´ar´any and Pach [3], Brown and Kasprzyk [4], and Lagarias and Ziegler [8]. Recall that an affine unimodular transformation T : Rd → Rd is a composition of the multiplication by a unimodular matrix A ∈ GL(d, Z) and a translation by an integer vector. We next reproduce the definition of the lattice size from [5], which now applies to a convex body P ⊂ Rd. Definition 0.1. Let X ⊂ Rd be a set with positive Jordan measure. The lattice size lsX (P ) with respect to X is the smallest integer l such that T (P ) is contained in

arXiv:1709.03454v4 [math.CO] 6 Oct 2020 the l-dilate of X for some affine unimodular transformation T . The corresponding matrix A is then said to compute lsX (P ). Note that when X = [0, 1] × Rn−1, the lattice size of P with respect to X is the lattice width w(P ), an important invariant in lattice geometry and its applications. In this paper we are studying ls∆(P ), the lattice size with respect to the standard d ±1 ±1 simplex ∆ ⊂ R . Let f ∈ k[x1 , . . . , xd ] be a Laurent polynomial over a field k and consider a hypersurface H in kd defined by f = 0. The Newton polytope P of f is the convex hull of the exponent vectors that appear in f. Given a unimodular

2010 Mathematics Subject Classification. 11H06, 52B20, 52C05, 52C07. Work of Soprunova and Tierney was partially supported by NSF Grant DMS-1156798. 1 2 ANTHONY HARRISON, JENYA SOPRUNOVA, AND PATRICK TIERNEY

a1i adi matrix A = (aij), a unimodular change of variables xi = u1 . . . ud transforms f into a Laurent polynomial with the Newton polytope A(P ). Therefore ls∆(P ) is the smallest possible total degree of the defining equation of H under unimodular changes of variables. Motivated by this observation, Schicho [10] provided an “onion skins” algorithm for mapping a lattice polygon P into l∆ for a small integer l. In [5] Castryck and Cools proved that this algorithm computes ls∆(P ). The algorithm is based on successively passing from an integer polygon to the convex hull of its interior lattice points. One can easily see that under this operation the lattice size drops by at least 3. It was shown in [5] that the lattice size ls∆(P ) drops exactly by 3, unless P belongs to a described list of exceptional cases. d Let  = [0, 1] be the unit cube. In the case d = 2 and a lattice polygon P , Castryck and Cools also provided in [5] an “onion skins” algorithm for computing the lattice ls(P ). While the “onion skins” algorithm is very visual, it involves listing the interior lattice points of P , which is time-consuming. It is also not feasible to generalize the “onion skins” algorithm to even dimension 3. In [6] the lattice size ◦ ls(P ) was recognized as a successive minimum of K = (P + (−P )) , the polar dual of the Minkowski sum of P with −P , the reflection of P in the origin. In dimension 2, the generalized Gauss Reduction algorithm of Kaib and Schnorr [7] computes the successive minima of an arbitrary origin-symmetric convex body K, and therefore it also computes the lattice size of the corresponding P , which can now be assumed to be an arbitrary convex body in R2. It is explained in [6] that in the case when P is a lattice polygon, the algorithm of Kaib and Schnorr outperforms the “onion skins” algorithm. The algorithm of Kaib and Schnorr was then extended in [6] to dimension 3. Hence, in particular, the generalized basis reduction algorithm of [6] 3 provides a fast way of computing ls(P ) for an arbitrary convex body P ⊂ R . Given a convex body P ⊂ R2, it was shown in [6] that if a basis (h1, h2) of Z2 is reduced with respect to K = (P + (−P ))◦ (see Definition 1.3) then matrix A with 1 2 rows h and h computes ls(P ). The main result of the current paper is formulated in Theorem 2.7 and Corollary 2.8, where we prove a similar statement for ls∆(P ). We define nls∆(P ) to be the smallest l ≥ 0 such that P is contained in l∆ after a transformation which is a composition of the multiplication by a unimodular matrix ±1 0  of the form and a lattice translation. We then show that if a lattice 0 ±1 1 2 basis (h , h ) is reduced (with respect to K) then ls∆(P ) = nls∆(AP ), where A is the matrix whose rows are h1 and h2. Therefore, the generalized Gauss reduction algorithm of Kaib and Schnorr can be used to find the lattice size ls∆(P ) of an arbitrary convex body P ⊂ R2. Not only this algorithm works for a much wider class of P ⊂ R2 than the “onion skins” algorithm, it is also faster in the case when P is a lattice polygon, as explained in [6]. It was further shown in [6] that if a lattice basis (h1, h2, h3) of Z3 is reduced then 1 2 3 matrix A with rows h , h , and h computes ls(P ). In Example 3 we demonstrate LATTICE SIZE OF PLANE CONVEX BODIES 3 that this is not the case for ls∆(P ) and therefore one cannot use basis reduction to compute ls∆(P ) in dimension 3.

1. Definitions Recall that GL(d, Z) is the set of square matrices of size d that are invertible over Z, that is, for A ∈ GL(d, Z) we have det A = ±1. A map T : Rd → Rd of the form T (x) = Ax + v, where x ∈ Rd, A ∈ GL(d, Z), and v ∈ Zd is called an affine unimodular map. Unimodular maps preserve the integer lattice Zd ⊂ Rd. For a lattice polytope P ⊂ Rd we will simply write AP for the image T (P ) of P under the linear transformation T : Rd → Rd defined by T (x) = Ax. A vector v ∈ Zd is primitive if its components are relatively prime.

Definition 1.1. Let P ⊂ Rd be a convex body. The lattice width of P in the direction of h ∈ Rd is defined by

wh(P ) = max h · x − min h · x, x∈P x∈P where h · x denotes the standard dot-product. The lattice width w(P ) of P is the d minimum of wh(P ) over all non-zero primitive vectors h ∈ Z . Let K := (P +(−P ))◦ be the polar dual of the Minkowski sum of P with −P , the reflection of P in the origin. Then K is origin-symmetric and convex and it defines a norm on Rd by khkK = inf{λ > 0 | h/λ ∈ K}. For details see, for example, [2]. We then have

◦ 1 ◦ khkK = inf{λ > 0 | h · x ≤ λ for all x ∈ K } = max h · x = wh(K ) = wh(P ). x∈K◦ 2 This, in particular, implies that the lattice width of P defines a convex function on d R . In what follows we will often write khkK or simply khk for the lattice width wh(P ). Let 0 denote the origin in Rd and let (e1, . . . , ed) be the standard basis of Rd. Next, let ∆ = conv{0, e1, . . . , ed} ⊂ Rd be the standard simplex. d Definition 1.2. The lattice size ls∆(P ) of a convex body P ⊂ R with respect to the standard simplex ∆ is the smallest l ≥ 0 such that P is contained in the l-dilate l∆ of the standard simplex ∆ after an affine unimodular transformation. A unimodular transformation T and the corresponding matrix A which minimize l are said to compute ls∆(P ). d In [6] the lattice size ls(P ) of a convex body P ⊂ R was recognized as a successive minimum of the corresponding K, which led to a fast lattice reduction algorithm for computing the lattice size with respect to the cube in dimensions 2 and 3. In this paper we will show that given a reduced basis, one can easily recover 2 ◦ ls∆(P ). Let P ⊂ R be a convex body and let K := (P + (−P )) . 4 ANTHONY HARRISON, JENYA SOPRUNOVA, AND PATRICK TIERNEY

Definition 1.3. We say that a basis (h1, h2) of the integer lattice Z2 ⊂ R2 is reduced (with respect to K) if

whi (P ) ≤ wh1±h2 (P ) for i = 1, 2.

If we use the notation khkK = wh(P ), this condition rewrites as i 1 2 kh kK ≤ kh ± h kK for i = 1, 2. 1 2 Usually one also requires that the basis vectors are ordered so that kh kK ≤ kh kK , but for our purposes it is convenient to omit this condition. The algorithm for finding a reduced basis of R2 with respect to an arbitrary origin- symmetric convex body K is explained and analyzed in [7]. A modified version of this algorithm is discussed in [6].

2. Reduced Basis Computes the Lattice Size Let P ⊂ R2 be a convex body.

Definition 2.1. We define the naive lattice size nls∆(P ) to be the smallest l such that T (P ) ⊂ l∆, where T : R2 → R2 is a combination of a matrix multiplication by a diagonal matrix A with entries ±1 on the main diagonal, and a lattice translation. Further, let

l1(P ) := max (x + y) − min x − min y, (x,y)∈P (x,y)∈P (x,y)∈P

l2(P ) := max x + max y − min (x + y), (x,y)∈P (x,y)∈P (x,y)∈P

l3(P ) := max y − min x + max (x − y), (x,y)∈P (x,y)∈P (x,y)∈P

l4(P ) := max x − min y + max (y − x). (x,y)∈P (x,y)∈P (x,y)∈P

Then nls∆(P ) is the smallest of these four values. 1 2 2 Definition 2.2. We say that basis (h , h ) of Z computes ls∆(P ) if for matrix A 1 2 with rows h , h we have ls∆(P ) = nls∆(AP ). Example 1. Let P = conv{(0, 0), (4, 1), (5, 2)}. Then, as demonstrated in the diagram, l1(P ) = 7, l2(P ) = 7, l3(P ) = 5, and l4(P ) = 5, and hence nls∆(P ) = 5. If

1 −4 we apply A = to P we get a triangle with the vertices (0, 0), (0, 1), (−3, 2). 0 1 LATTICE SIZE OF PLANE CONVEX BODIES 5

Then l1(AP ) = 4, l2(AP ) = 3, l3(AP ) = 5, and l4(AP ) = 4, so nls∆(AP ) = 3. Since P has a lattice point inside, while 2∆ does not, it is impossible to unimodularly map 1 −4 P inside 2∆. We conclude that ls (P ) = 3, and A = and the lattice basis ∆ 0 1 ((1, −4), (0, 1)) compute the lattice size of P . We first record a few straight-forward observations.

2 Lemma 2.3. For h ∈ R and A ∈ GL(2, Z) we have wh(AP ) = wAT h(P ). There- 1 2 fore, if the rows of A are h and h , then we1 (AP ) = wh1 (P ) and we2 (AP ) = wh2 (P ).

Proof. For any h ∈ R2 we have T T wh(AP ) = max h · (Ax) − min h · (Ax) = max(A h) · x − min(A h) · x = wAT h(P ). x∈P x∈P x∈P x∈P 

Lemma 2.4. (a) If P is reflected in the line x = y, then l1 and l2 do not change, while l3 and l4 switch with each other. (b) If P is reflected in the x-axis, then l1 switches with l3, while l2 switches with l4. (c) If P is reflected in the y-axis, then l1 switches with l4, while l2 switches with l3. (d) The naive lattice size nls∆(P ) is preserved under the reflection in the line y = x as well as under the reflections in the x- and y-axis. Lemma 2.5. (a) a b  l1 P = max ((a + c)x + (b + d)y) c d (x,y)∈P − min (ax + by) − min (cx + dy); (x,y)∈P (x,y)∈P

a b  c d  (b) l P = l P ; 1 c d 1 a b (c) a b  −(a + c) −(b + d)  l P = l P 1 c d 1 c d  a b   = l P . 1 −(a + c) −(b + d)

Proof. First equality of part (c) is equivalent to claiming that l1(AP ) = l1(SAP ), a b −1 −1 where A = and S = , so replacing AP with P , it is enough to c d 0 1 6 ANTHONY HARRISON, JENYA SOPRUNOVA, AND PATRICK TIERNEY show that l1(SP ) = l1(P ). We have

l1(SP ) = max (−x) − min (−x − y) − min y (x,y)∈P (x,y)∈P (x,y)∈P

= max (x + y) − min x − min y = l1(P ). (x,y)∈P (x,y)∈P (x,y)∈P  a b Lemma 2.6. Suppose that for A = we have l := l (AP ) < l. Then the c d 1 1 lattice width of P in the three directions, (a, b), (c, d), and (a + c, b + d), is less than l.

Proof. We have AP ⊂ l1∆, which implies that we1 (AP ) ≤ we1 (l1∆) = l1. Hence using Lemma 2.3 we get

w(a,b)(P ) = we1 (AP ) ≤ l1 < l. The same argument applied to directions e2 and e1 + e2 gives the remaining two conlcusions.  ◦ Theorem 2.7. Let P ⊂ R2 be a convex body and let K = (P + (−P )) . If the 1 2 standard basis (e , e ) is reduced with respect to K then ls∆(P ) = nls∆(P ).

Proof. We will use the notation khk = khkK for wh(P ). First, shift P so that min x = min y = 0 and denote (x,y)∈P (x,y)∈P

l := l1(P ) = max (x + y). (x,y)∈P Next, denote k = max x; m = max y; and s = min (x + y). Note that since (x,y)∈P (x,y)∈P (x,y)∈P there is a point of the form (a, 0) in P we get s ≤ a ≤ k. Similarly, s ≤ m. We have ke1k = k, ke2k = k, and ke1 + e2k = l − s. Since the standard basis is reduced, we get l − s = ke1 + e2k ≥ ke1k = k and l − s = ke1 + e2k ≥ ke2k = m. This implies that l ≥ 2s since k ≥ s and m ≥ s. Hence 1 2 (2.1) 2ke + e k = 2l − 2s = l + (l − 2s) ≥ l ≥ nls∆(P ). Note that we also have k + m ≥ l since P ⊂ [0, k] × [0, m] ⊂ (k + m)∆. We record the obtained relations for future use (2.2) l ≥ k + s, l ≥ m + s, l ≥ 2s, k + m ≥ l. Pick a primitive direction (a, b) ∈ Z2. Our goal is to show that kae1 + be2k ≥ nls∆(P ) for many such (a, b). Note that if we have a reduced basis, we can flip its vectors and their signs and the obtained basis will also be reduced. Also, as we observed in Lemma 2.4, the naive lattice size nls∆(P ) is invariant with respect to the reflections in the coordinate axes and the origin. Hence without loss of generality we can assume that a ≥ b ≥ 0. Using the triangle inequality we get kae1 + be2k + (a − b)ke2k = kae1 + be2k + k(a − b)e2k ≥ ake1 + e2k. LATTICE SIZE OF PLANE CONVEX BODIES 7

On the other hand, since (e1, e2) is reduced we get kae1 + be2k + (a − b)ke2k ≤ kae1 + be2k + (a − b)ke1 + e2k. Hence for b ≥ 2, combining the above two lines and (2.1), we get

1 2 1 2 1 2 kae + be k ≥ bke + e k ≥ 2ke + e k ≥ nls∆(P ). 1 2 We have checked that kae + be k(P ) ≥ nls∆(P ) for all primitive directions (a, b) with min{|a|, |b|} ≥ 2. Without loss of generality we can assume that k ≤ m, that is, ke1k ≤ ke2k. If b ≥ a = 1 then bm = bke2k ≤ ke1 + be2k + ke1k = ke1 + be2k + k, so for b ≥ 3 we get

1 2 ke + be k ≥ bm − k ≥ (b − 1)m ≥ 2m ≥ k + m ≥ l ≥ nls∆(P ). 1 2 We have checked that under the assumption k ≤ m we have kae +be k ≥ nls∆(P ) for all primitive (a, b) except, possibly, for the ones in the set

R = {(a, ±1), (±1, 0), (±1, ±2) | a ∈ Z}. It follows from Lemma 2.6 that it remains to show that if the rows and the row sum of a unimodular matrix A are in R then l1(AP ) ≥ nls∆(P ). a 1 Let first A = with a ≥ 0. Then 1 0

l1(AP ) = max ((a + 1)x + y) − min (ax + y) − min (x). (x,y)∈P (x,y)∈P (x,y)∈P

Since P ⊂ l∆ has a point on the segment that joins points (k, l − k) and (l − m, m) we have max ((a+1)x+y) ≥ min{(a+1)k+(l−k), (a+1)(l−m)+m} = (a+1)(l−m)+m, (x,y)∈P where we used k + m ≥ l from (2.2). Using this together with min (ax + y) ≤ as (x,y)∈P we get l1(AP ) ≥ (a + 1)(l − m) + m − as. Then (a + 1)(l − m) + m − as ≥ l is equivalent to a(l − m) ≥ as, which holds true since a ≥ 0 and l ≥ m + s. 8 ANTHONY HARRISON, JENYA SOPRUNOVA, AND PATRICK TIERNEY

1 1 For A = we get 0 1

l1(AP ) = max (x + 2y) − min (x + y) − min y (x,y)∈P (x,y)∈P (x,y)∈P

= max (x + 2y) − s ≥ 2m − s ≥ k + m − s = l2(P ) ≥ nls∆(P ). (x,y)∈P

Suppose that we showed that for a matrix A we have l1(AP ) ≥ nls∆(P ) whenever (e1, e2) is reduced with respect to P . Let Q be the reflection of P in the x-axis. Then since (e1, e2) is also reduced with respect to Q, we have also shown that l1(AQ) ≥ nls∆(Q). This implies that

−a b  a b −1 0  l P = l P = l (AQ) ≥ nls (Q) = nls (P ). 1 −c d 1 c d 0 1 1 ∆ ∆

Hence if we proved that l1(AP ) ≥ nls∆(P ) for matrix A, then the same conclusion holds true for a matrix that can be obtained from A by flipping one or both column signs. Our next goal is to show that unimodular matrices whose rows and the row sum 1 1 a 1 are in the set R reduce to either A = or with a ≥ 0 by flipping 0 1 1 0 column signs and using reductions described in part(c) of Lemma 2.5. Since (±1, 0) and (±1, ±2) cannot be the two rows of a unimodular matrix, we can assume that first row of A is of the form (a, ±1). If now the second row is of the  a 1   a −1 form (±1, ±2), that is, A = or A = (the row sum also has to ±1 −2 ±1 2 be in R), using Lemma 2.5, we can change the second row to one of the form (b, ±1) a ±1 for some b ∈ and get a matrix of the form for some a, b ∈ . Z b ±1 Z  a ±1 If the second row of A is of the form (±1, 0) we get A = for some ±1 0 a 1 a ∈ . Switching column signs we obtain a matrix of the form , where a ∈ . Z 1 0 Z We now identify unimodular matrices both of whose rows are of the form (a, ±1). a 1  Let first A = for some a, b ∈ . Then the sum of rows (a + b, 0) is in R we b −1 Z  a 1  get a + b = ±1. Hence A = for some a ∈ and using Lemma 2.5 we ±1 − a −1 Z  a 1 can replace A with , and switching the sign of the first column, if necessary, ±1 0 a 1 we get a matrix of the form with a ∈ . 1 0 Z LATTICE SIZE OF PLANE CONVEX BODIES 9

a 1 Next let A = for a, b ∈ . Then a + b = ±1 and a − b = ±1, where b 1 Z for the second condition we used det A = ±1. Switching rows if necessary we get 1 1 −1 1 a −1 1 −1 A = or . If A is of the form we get A = or 0 1 0 1 b −1 0 −1 −1 −1 1 1 . All of these matrices reduce to after we flip column signs. 0 −1 0 1 −a 1 a − 1 −1 Next note that using Lemma 2.5 we can pass from A = to 1 0 1 0 a − 1 1 and switching the sign of the second column we pass to the matrix . This 1 0 a 1 shows that matrices of the form with a ∈ reduce to matrices of the same 1 0 Z form with a ≥ 0, for which we have checked that l1(AP ) ≥ nls∆(P ). We have checked that for all unimodular A we have l1(AP ) ≥ nls∆(P ), which implies ls∆(P ) = nls∆(P ).  Corollary 2.8. Let P ⊂ R2 be a convex body. If the basis (h1, h2) of Z2 is reduced 1 with respect to P then ls∆(P ) = nls∆(AP ), where A is a matrix whose rows are h and h2. Proof. By Lemma 2.3 we have

we1 (AP ) = wh1 (P ), we2 (AP ) = wh2 (P ), and we1±e2 (AP ) = wh1±h2 (P ), and it follows that the standard basis (e1, e2) is reduced with respect to AK. By Theorem 2.7 this implies that ls∆(P ) = ls∆(AP ) = nls∆(AP ).  The algorithm for finding a reduced basis is explained and analyzed in [6] and [7]. In each iteration of the algorithm, starting with basis (h1, h2) with kh1k ≤ kh2k, one finds m that minimizes kmh1 + h2k. If kmh1 + h2k ≥ kh1k then the basis (h1, mh1 + h2) is reduced. Otherwise, the next iteration is applied to the basis (mh1 + h2, h1). In the example below we find a reduced basis and compute the lattice size. Example 2. Let P = conv{(0, 0), (3, 5), (7, 9), (8, 12)} and let h1 = e1, h2 = e2. 1 2 Then ke kP = 8 and ke kP = 12. For m ∈ Z we have 1 2 kme + e kP = max{0, 3m + 5, 7m + 9, 8m + 12} − min{0, 3m + 5, 7m + 9, 8m + 12}. 1 2 1 2 Hence kme + e kP ≥ 8m + 12 ≥ 12 for m ≥ 0 and kme + e kP ≥ −7m − 9 ≥ 5 1 2 1 for m ≤ −2. Also, k − e + e kP = 4 < ke kP = 5 and hence we should pass to the −1 1 polygon AP , where A = . We get AP = conv{(0, 0), (2, 3), (2, 7), (4, 8)}. 1 0 1 2 1 2 Now the minimum of kme + e kAP is achieved at m = −2 and k − 2e + e kAP = 4. 1 2 1 Since k − 2e + e kAP ≥ ke kAP the algorithm terminates at this step. Hence matrix 10 ANTHONY HARRISON, JENYA SOPRUNOVA, AND PATRICK TIERNEY

−1 1   1 0 −1 1 B = = computes the lattice size and basis ((−1, 1), (3, −2)) 3 −2 −2 1 1 0 is reduced with respect to P . We get BP = conv{(0, 0), (2, −1), (2, 3), (4, 0)}. We conclude that ls∆(P ) = nls∆(BP ) = min{8, 6, 5, 6} = 5. 3. Counterexample in the 3-space. 3 For a convex body P ⊂ R define nls∆(P ) to be the smallest l such that P is con- ±1 0 0  tained in l∆ after a transformation which is a composition of A =  0 ±1 0  0 0 ±1 and a lattice translation. We also define

l1(P ) := max (x + y + z) − min x − min y − min z. (x,y,z)∈P (x,y,z)∈P (x,y,z)∈P (x,y,z)∈P ◦ Let P ⊂ R3 be a convex body and let K = (P + (−P )) . Definition 3.1. A basis (h1, h2, h3) of Z3 ⊂ R3 is reduced with respect to K if (1) kh1k ≤ kh2k ≤ kh3k, (2) kh1 ± h2k ≥ kh2k, and (3) kmh1 + nh2 + h3k ≥ kh3k for all m, n ∈ Z. Note that condition (2) is equivalent to a seemingly stronger requirement that kmh1 + h2k ≥ kh2k for all m ∈ Z. Assume that kh1 ± h2k ≥ kh2k. Then for m ∈ Z we have kmh1 + h2k + (|m| − 1)kh2k ≥ |m|kh1 + sgn(m)h2k ≥ |m|kh2k, which implies that kmh1 + h2k ≥ kh2k. Hence the above definition is a natural generalization of Definition 1.3 It is shown in [6] that if a basis (h1, h2, h3) is reduced then matrix A whose rows 1 2 3 are h , h , and h computes ls(P ), which led to a fast algorithm for computing ls(P ) in dimension 3. Hence it is natural to ask whether for such A we have ls∆(P ) = nls∆(AP ). We can also relax this question and ask whether there exists a 1 2 3 reduced basis (h , h , h ) such that for the corresponding matrix A we have ls∆(P ) = nls∆(AP ). The answer to both of these questions is negative, as the example below shows. Example 3. Let P = conv{(1, 1, 2), (4, 4, 4), (0, 2, 2), (3, 0, 3), (4, 3, 0)} ⊂ R3. Then 3 we1 (P ) = we2 (P ) = we3 (P ) = 4. Let’s check that for primitive (a, b, c) ∈ Z we have w(a,b,c)(P ) > 4 unless one of a, b, and c is ±1 and the other two numbers are 0. We have

w(a,b,c)(P ) = max{a + b + 2c, 4a + 4b + 4c, 2b + 2c, 3a + 3c, 4a + 3b} − min{a + b + 2c, 4a + 4b + 4c, 2b + 2c, 3a + 3c, 4a + 3b}. We can assume that c ≥ 0. Let first c ≥ 1. Then if b ≥ 1 we have

w(a,b,c)(P ) ≥ (4a + 4b + 4c) − (4a + 3b) = b + 4c ≥ 5. LATTICE SIZE OF PLANE CONVEX BODIES 11

For b ≤ −1 and a ≥ 1, or b = 0 and a ≥ 2 we have

w(a,b,c)(P ) ≥ (3a + 3c) − (a + b + 2c) = 2a − b + c ≥ 5. For b ≤ −1 and a ≤ 0, or b = 0 and a ≥ −2 we get

w(a,b,c)(P ) ≥ (3a + 3c) − (4a + 3b) = −a − 3b + 3c ≥ 5.

If a = 1 and b = 0 then w(a,b,c)(P ) ≥ (4a + 4b + 4c) − (2b + 2c) = 4 + 2c ≥ 6. If a = −1 and b = 0 then w(a,b,c)(P ) ≥ (a + b + 2c) − (4a + 3b) = 2 + 2c ≥ 5. Next, let c = 0, so we can assume that a ≥ 0. If a ≥ 1 and b ≥ 1 we have w(a,b,c)(P ) ≥ (4a + 4b) − (2b) ≥ 6. If a ≥ 1 and b ≤ −1 we have w(a,b,c)(P ) ≥ 3a − 2b ≥ 5. If a = 0 and b ≥ 1 then w(a,b,c)(P ) ≥ 4b ≥ 8 unless b = 1. We have checked that the only primitive directions in which the lattice width of P is at most 4 are ±e1, ±e2, and ±e3 and the lattice width of P in these directions is 4. Hence the bases that one can obtain from (±e1, ±e2, ±e3) by switching the order of the vectors and flipping their signs are reduced. We next show that there are no other reduced bases here. Let now (h1, h2, h3) be reduced. It is shown in Theorem 3.3 in [6] that kah1 +bh2 + ch3k ≥ kh3k for all a, b, c ∈ Z with nonzero c, except, possibly, for one primitive direction ±(ah1 + bh2 + ch3) with |a| = |b| = 1 and |c| = 2. Hence if kh3k ≥ 5 we can conclude that two standard vectors, say, e1 and e2, belong to the (h1, h2)-plane. Since, as was shown in Theorem 2.3 in [6], for a, b ∈ Z we have kah1 + bh2k ≥ kh2k for all nonzero b, we can conclude that bases (h1, h2) and (e1, e2) coincide up to changing the order of vectors and flipping their signs. Since ke3k = 4, this would imply that e3 = ae1 + be2 + ch3, where |a| = |b| = 1 and |c| = 2, but then the change of basis matrix from (h1, h2, h3) to the standard basis would have determinant 2 or −2, which is impossible. We conclude that kh3k = 4 and (h1, h2, h3) is obtained from the standard basis by switching the order of the vectors and flipping their signs. Next, one can easily check that nls∆(P ) = 8. Using the obtained description of reduced bases we conclude that for matrix A whose rows form a reduced basis we −1 0 0 also have nls∆(AP ) = 8. For B =  0 −1 0 we get 0 1 1 BP = conv{(−1, −1, 3), (−4, −4, 8), (0, −2, 4), (−3, 0, 3), (−4, −3, 3)}.

Hence l1(BP ) = 2 − (−4) − (−4) − 3 = 7 and we conclude that ls∆(P ) ≤ 7, so there is no reduced basis here such that for the corresponding matrix A we have ls∆(P ) = nls∆(AP ). Acknowledgments. Work of Soprunova and Tierney was partially supported by NSF Grant DMS-1156798.

References [1] V. Arnold, Statistics of integral convex polygons, Functional Analysis and Its Applications 14(2), 1-3 (1980) 12 ANTHONY HARRISON, JENYA SOPRUNOVA, AND PATRICK TIERNEY

[2] A. Barvinok, Integer Points in Polyhedra, Zurich Lectures in Advanced Mathematics, ISBN: 9783037190524 (2008) [3] I. B´ar´any and J. Pach, On the number of convex lattice polygons, Combinatorics, Probability and Computing 1, Issue 4 (1992) [4] G. Brown, A. Kasprzyk, Small polygons and toric codes, Journal of Symbolic Computation, 51 p. 55 (2013) [5] W. Castryck , F. Cools, The lattice size of a lattice polygon, Journal of Combinatorial Theory Series A 136, Issue C, 64-95 (2015) [6] A. Harrison, J. Soprunova, Lattice Size of 2D and 3D polytopes with respect to the unit cube, preprint, arXiv:1709.03451 (2020) [7] M. Kaib, C. Schnorr, The Generalized Gauss Reduction Algorithm, Journal of Algorithms 21(3) 565-578 (1996) [8] J. Lagarias, G. Ziegler, Bounds for lattice polytopes containing a fixed number of interior points in a sublattice, Canadian Journal of Mathematics 43(5), 1022-1035 (1991) [9] L. Lov´asz,H. Scarf The Generalized Basis Reduction Algorithm, Mathematics of Operations Research 17, Issue 3, 751-764 (1992) [10] J. Schicho, Simplification of surface parametrizations – a lattice polygon approach, Journal of Symbolic Computation 36(3-4), 535-554 (2003).

Department of Mathematics, Kent State University, Summit Street, Kent, OH 44242, USA Email address: [email protected]

Department of Mathematics, Kent State University, Summit Street, Kent, OH 44242, USA Email address: [email protected] URL: http://www.math.kent.edu/~soprunova/

Department of Mathematics, University of Washington, Box 354350, Seattle, WA 98195, USA Email address: [email protected]