ALGORITHMS FOR COMPUTING THE LATTICE SIZE

A dissertation submitted to Kent State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy

by

Anthony Harrison

August 2018 c Copyright All rights reserved Except for previously published materials Dissertation written by Anthony Harrison B.S., Texas State University–San Marcos, 2010 M.S., Texas State University–San Marcos, 2013 Ph.D., Kent State University, 2018

Approved by

Jenya Soprunova , Chair, Doctoral Dissertation Committee

Ivan Soprunov , Members, Doctoral Dissertation Committee

Mark Lewis

Mikhail Chebotar

Feodor Dragan

Austin Melton

Accepted by

Andrew Tonge , Chair, Department of Mathematical Sciences

James L. Blank , Dean, College of Arts and Sciences TABLE OF CONTENTS ...... iii

LIST OF FIGURES ...... iv

ACKNOWLEDGMENTS ...... v

1 INTRODUCTION ...... 1

2 LATTICES AND POLYTOPES ...... 2 2.1 Affine and Convex Geometry ...... 2 2.2 Lattices ...... 3 2.3 Polytopes ...... 4 2.4 Lattice Width ...... 6 2.5 Newton Polytopes ...... 7

3 LATTICE SIZE ...... 9 3.1 Introduction ...... 9 3.2 Algebraic Geometry and the Lattice Size ...... 10 3.3 Brute-force algorithms ...... 12 3.4 The Onion Skins Algorithm ...... 13

4 COMPUTING THE LATTICE SIZE WITH RESPECT TO THE CUBE ...... 15 4.1 Definitions ...... 15 4.2 Lattice size of polygons with respect to the unit square ...... 16 4.3 Lattice size in dimension 3 ...... 19

5 COMPUTING THE LATTICE SIZE WITH RESPECT TO THE SIMPLEX ...... 29 5.1 Definitions ...... 29 5.2 Main Results ...... 30

iii List of Figures

Figure 1: Width in the direction a ...... 6 Figure 2: Newton polytope of f ...... 7

Figure 3: Lattice size with respect to the standard simplex ...... 9 Figure 4: Convex hull of interior lattice points ...... 13

Figure 5: Lattice size with respect to the unit square ...... 15

Figure 6: Examples of s, u, and t ...... 31 Figure 7: Example of line segments and simplexes ...... 32 Figure 8: Region determined by special case ...... 37 Figure 9: Directions with large width ...... 38

iv Acknowledgments

I would like to thank Jenya Soprunova for her mentorship and support during the research for this project. I would also like to thank Ivan Soprunov for the time he spent teaching and working with me. I am also grateful to the faculty and graduate students in Kent State University’s Math Department for providing a supportive and helpful environment; I had many questions and they were willing to listen and offer their insight.

v Chapter 1

Introduction

n In this dissertation, we study the lattice size problem. For a lattice polytope P ⊂ R and a n region X ⊂ R of positive Jordan measure, we are interested in determining the smallest integral dilate of X that contains P after an affine unimodular transformation:

lsX (P) = min{l ∈ N : T(P) ⊂ lX,T ∈ AGL(n,Z)} where lX = {lx : x ∈ X} and AGL(n,Z) denotes the group of affine unimodular transformations n which act on R (see Chapter 2 and 3 for more details). This problem has been encountered in a number of contexts [1, 2, 14, 16]. There are algorithms [6, 5] that can be used to determine the lattice size, but each of them requires enumerating lattice points in some region. Such an operation is computationally expensive. We will provide new algorithms [11, 12] determine the lattice size for certain choices of X which avoid lattice point enumeration. This dissertation begins with relevant background information on geometry and polytopes (Chapter 2). Chapter 3 gives a detailed history of the lattice size problem and explains some of its connections to algebraic geometry and coding theory. We also give a brief inventory of other algorithms that compute the lattice size. Chapter 4 and 5 constitute the main content of this dis- sertation. These chapters detail new fast algorithms to compute the lattice size with respect to the two- and three-dimensional unit cube and the standard two-dimensional simplex.

1 Chapter 2

Lattices and Polytopes

2.1 Affine and Convex Geometry

In this section, it will be convenient to use affine spaces and convex geometry so we collect some definitions here. An affine space is similar to a vector space, but lacks a distinguished origin and has a different algebraic structure. For example, the line L given by y = 1 − x in R2 can be endowed with the structure of an affine space. We note that this set is not closed under addition. However, if we subtract any pair of points a,b ∈ L, we get a translation vector. Such a vector v = a − b determines an affine transformation f : L → L given by f (x) = x + v. The set of all such translations is in fact a vector space. The dimension of an affine space is the dimension of this vector space. Since affine spaces are not closed under all linear combinations, we need a new notion to generate affine spaces. We define an affine combination of a set of points x1,...,xk to be a linear combination λ1x1 + ··· + λkxk such that each λi is a real number and λ1 + ··· + λk = 1. Suppose n S ⊂ R . The affine hull of S is the set

( k k ) aff(S) = ∑ λixi : k > 0,xi ∈ S, ∑ λi = 1 . i=1 i=1

This set is smallest affine space which contains S.

A convex combination is a linear combination λ1x1 +···+λkxk such that λi ≥ 0 and λ1 +···+

λk = 1. A set that is closed under convex combinations is a convex set. For any convex set X, if x,y ∈ X, then the line segment between x and y is also in X. This idea gives an intuitive way to think about convex sets.

2 n Definition The convex hull of a set S ⊂ R is the region

( k k ) conv(S) = ∑ λixi : k > 0,xi ∈ S, ∑ λi = 1,λi ≥ 0 . i=1 i=1

The set S generates its convex hull. This terminology is appropriate since conv(S) is the smallest convex set that contains S [3].

2.2 Lattices

n n A lattice is a discrete subgroup of R that is isomorphic to Z . In this dissertation, we will only n need to work with the lattice Z for an appropriate choice of n. This section focuses on morphisms n n of R that preserve Z . Below we denote the n × n matrices with entries in Z by Mn(Z).

n n Definition A unimodular transformation is a linear transformation f : R → R such that f (v) =

Av for some matrix A ∈ Mn(Z) with det(A) = ±1.

An important property of these transformations is that they map lattice vectors, i.e. vectors in the

n n lattice Z , to lattice vectors. In fact, these maps are automorphisms of Z [17]. Among these maps are reflections through the coordinate hyperplanes and certain 90◦ rotations. If A determines a unimodular transformation, then so does A−1. We call the class of matrices which yield such transformations unimodular and denote them by GL(n,Z). There is a larger class of maps which n also preserves Z .

n n Definition An affine unimodular transformation f : R → R is a unimodular transformation com- n posed with a translation by a lattice vector. If A ∈ GL(n,Z) and v ∈ Z , then the following is such a transformation: f (x) = Ax + v.

n We denote the group of affine unimodular transformations on R by AGL(n,Z).

3 2.3 Polytopes

A polytope is a particular region of Euclidean space with two equivalent characterizations [17]:

1. A convex hull of finitely many points.

2. A bounded intersection of finitely many half spaces.

We will focus on the first characterization.

Definition A polytope P is the convex hull of finitely many points {s1,...,sm}. The minimal set V such that conv(V) = P is called the vertex set of P. Elements of V are called vertices. The dimension of a polytope is the dimension of the affine hull of P.

This formal definition covers the most commonly encountered polytopes (called polygons in di- mension two) such as squares, triangles, cubes, tetrahedra, etc.

n n Definition A lattice polytope is a polytope in R whose vertices are in Z .

There are a number of operations on lattice polytopes relevant to this dissertation. Suppose

n P ⊂ R is a polytope. We first discuss int(P), the interior of P. If P is full dimensional, dimP = n, n then the interior takes on its typical meaning for a closed set in R . In other situations, it will be more useful to use the relative interior, relint(P): the interior of P in its affine span.

n We can translate P by any vector v ∈ R : P + v = {x + v : x ∈ P}. We can also scale a polytope by some factor r ∈ R and denote this by rP = {rx : x ∈ P}. Affine unimodular transformations act on the set of lattice polytopes. In much of what follows, we will work with the equivalence relation induced by this action on lattice polytopes. Let us first

n consider very simple lattice polytopes: line segments with endpoints in Z . Such a line segment is called a primitive lattice segment if the only lattice points it contains are its endpoints. We say two segments are unimodularly equivalent if there is an affine unimodular transformation that

maps one segment to the other. It is easy to see that any two primitive lattice segments in R2 are

4 unimodularly equivalent. Suppose (a,b) is a primitive lattice segment in R2. Then gcd(a,b) = 1 so there are integers s and t such that as + bt = 1. Then

     −b a a 0         =  . s t b 1

Since unimodular transformations are invertible, we can map any primitive lattice segment to any other primitive lattice segment. In fact, this is true in any dimension.

Proposition 2.3.1. Every pair of primitive lattice segments are unimodularly equivalent.

n Proof. It suffices to show that any primitive lattice segment in R is equivalent to en, the nth n standard basis vector. Consider a primitive lattice segment with endpoints a,b ∈ Z . Define a n vector x = a − b ∈ Z . It is clear that conv{a,b} is unimodularly equivalent to conv{0,x}. We will assume that x has no zero entries, since the presence of zeroes would allow us to work in a lower dimension. Using permutation matrices, if necessary, we can ensure that x1 > x2 for x = (x1,x2,...,xn). Let gcd(x1,x2) = y and suppose that the sequence of quotients when applying   0 1   the Euclidean algorithm is q1,q2,...,qr for some positive integer r. Let Qi =   for i = 1 −qi 1,...,r and let I denote the n − 2 × n − 2 identity matrix. Consider the calculation

   0  x1 x1             x2  y      Qr 0 Qr−1 0 Q1 0       ... x  = x       3  3 0 I 0 I 0 I      .   .   .   .      xn xn

0 where x1 is some integer. Each Qi is unimodular so each block matrix will be unimodular. We can iterate this process by appropriately placing the Qi blocks along the diagonal and permuting coordinates as necessary. Since we must have that gcd(x1,...,xn) = 1, the final iteration will yield a 1 in the last coordinate.

5 Suppose our new vector is x = (x1,...,xn−1,1). Then

     1 0 ... 0 −x1 x1 0           0 1 −x2  x2  0      . . .  .  . . .. .  .  = ..           .     . 1 −xn−1xn−1 0      0 ...... 0 1 1 1

n For lattice polytopes, we also have a notion of equivalence. If P,Q ⊂ R are lattice polytopes, we say that P is unimodularly equivalent to Q if there is an f ∈ AGL(n,Z) such that the image of P under f is Q.

2.4 Lattice Width

The lattice width of a polytope is an important concept that has been intensely studied [7, 10, 9, 8]. It is used in algorithms to solve integer programming problems. We first define a parameterized

n version of the notion. We say that a vector a ∈ Z is a primitive direction if conv{0,a} is a n primitive lattice segment. The width of a polytope P in a primitive direction a ∈ Z is the number wa(P) = maxx∈Pha,xi − minx∈Pha,xi where h,i denotes the standard inner product.

a

Figure 1: Width in the direction a

6 n Definition The lattice width of a polytope P ⊂ R is

n ~ w(P) = min{wa(P) : a ∈ Z \{0} is a primitive direction}

Alternatively, the lattice width can be defined using unimodular transformations: w(P) = min{we1 (AP) :

A ∈ GL(n,Z)} where e1 denotes a standard basis vector.

2.5 Newton Polytopes

Lattice polytopes arise naturally in the study of Laurent polynomials, i.e. polynomials where indeterminates may have negative exponents. Suppose we have a polynomial f = x3y + xy−2 + xy + 1. Its exponents can be represented by the integer vectors {(3,1),(1,−2),(1,1),(0,0)}. The

1

1 2 3

-1

-2

Figure 2: Newton polytope of f convex hull of these vectors will be a lattice polytope. We introduce some notation to more easily

n m n mi communicate this concept. For any m = (m1,...,mn) ∈ Z , define x = ∏i=1 xi . s m j n Definition Let f = ∑ a jx be a Laurent polynomial where a j 6= 0 and m j ∈ Z for some n. Its j=1 Newton polytope is conv{mi : 1 ≤ i ≤ s}.

The Newton polytope is one way to define an analog of the degree for Laurent polynomials. To see this, consider f = 3x2 + x = x(3x + 1). Since indeterminates are units in the Laurent polynomial

± ring C[x 1], we should not assign such a polynomial degree 2 as we would in the usual polynomial ring. If we take the convex hull of its exponent vectors, we get the segment [1,2] ⊂ R which is of

7 length 1. This length corresponds to the number of distinct non-zero roots. This use of Newton polytopes to determine the number of roots has a far-reaching generalization called the Bernshtein- Kouchnirenko theorem [4, 13].

8 Chapter 3

Lattice Size

3.1 Introduction

We begin by defining the central notion of this chapter:

n Definition The lattice size of a polytope P ⊂ R with respect to a set X of positive Jordan measure is

lsX (P) = min{l ≥ 0 : T(P) ⊂ lX,T ∈ AGL(n,Z)}.

Suppose X = ∆ is the standard simplex in 2-dimensions, i.e. the convex hull of (0,0), (1,0), and (0,1). Consider the polytope P = conv{(1,0), (2,2), (0,1)}. Its lattice size with respect to the standard simplex is in fact 3. To see this, we can reflect it across the origin and translate it to the

first quadrant. After this affine unimodular transformation, the polytope will fit inside 3∆. Since there is the lattice point (1,1) in the interior of P, we cannot hope to fit P inside of a smaller dilate via an affine unimodular transformation (because such maps preserve the relationship between lattice points).

Figure 3: Lattice size with respect to the standard simplex

9 The lattice size generalizes the well-studied lattice width parameter. There are good algorithms to compute the lattice width in any dimension [7]. The lattice size, on the other hand, is less well understood and the only general algorithms are rather slow. For specific choices of X, there are better algorithms [12, 11]; some of these are detailed in this dissertation. Problems involving the lattice size were first studied by Arnold [1]. He considered optimally fitting a lattice polytope into a square up to a unimodular transformation. Bar´ any´ and Pach used a similar idea in their paper [2]. Some recent interest has been driven by results of Schicho [16].

3.2 Algebraic Geometry and the Lattice Size

Schicho was interested in how to simplify rational functions defining a surface [15] (i.e. a two-

n dimensional algebraic variety). We let P denote n-dimensional projective space over a field k. He n defined a parameterization of a surface to be a map f : P2 → P given by

(s : t : u) 7→ (F0 : F1 : ··· : Fn)

where Fi are homogeneous polynomials of degree d in k[s,t,u] for some algebraically closed field k. The parameterized surface is then S = im f and it has multiple parameterizations. Schicho’s goal was to find a parameterization whose defining polynomials had the smallest degree possible for a given surface S. His 2002 paper showcased an algorithm that, starting with a particular parameterization, itera- tively found parameterizations of smaller degree. He proved that the output of his algorithm was no worse than twice the optimal degree. A year later, with input from others familiar with his work, Schicho published a new paper [16] about his algorithm in the toric surface case. Since projective toric varieties are closely related to lattice polytopes, the steps of Schicho’s algorithm could be interpreted as operations on lattice polytopes. With this view, Schicho’s algorithm gave an effective means for computing the lattice size of a polygon with respect to the standard simplex. (It works only for polygons because of the

10 two-dimensional nature of Schicho’s algorithm, which applies to surfaces. The two-dimensional

simplex is related to the toric variety structure of P2, the domain of a parameterization.) Castryck and Cools expanded upon Schicho’s results and provided a different problem closely associated with his algorithm and the lattice size. In their setup, they framed the problem in terms

of birational geometry. Let C be a curve in the 2-dimensional algebraic torus, T2 = (k \{0})2, over some field k. Their question was: what is the smallest possible degree of a curve that is

2 birationally equivalent to C? This smallest degree is denoted S2(C). A curve in T is defined by a Laurent polynomial f ∈ k[x±1,y±1] and its degree is defined below. Let ∆ denote the standard two-dimensional simplex.

Definition The total degree of a Laurent polynomial f with Newton polytope P is the smallest

integer l such that P + v ⊂ l∆ for some vector v.

If we now consider a restricted class of birational maps called toric automorphisms, then we have a

one-to-one correspondence with unimodular transformations. Suppose that ϕ : T2 → T2 is a toric automorphism. This induces a map on the coordinate rings ϕ∗ : k[x±1,y±1] → k[x±1,y±1] that is associated to a unimodular transformation. To get such a transformation A, we need only check

the images of x and y under ϕ∗. If ϕ∗(x) = xayc and ϕ∗(y) = xbyd, then

  a b   A =   c d

and this will be invertible because ϕ is. To see that its determinant must be ±1, one only needs that its inverse must also be an integral matrix. Now, ϕ∗ will induce a map of Newton polytopes. For a polynomial f ∈ k[x±1,y±1], its Newton polytope P will map to A(P). Thus, computing

the lattice size of P with respect to the simplex ∆ will yield an upper bound for S2(C) where 2 C = {t ∈ T : f (t1,t2) = 0}. The most recent paper involving the lattice size uses it to understand linear error-correcting codes. Kasprzyk and Brown [5] were interested in “champion” codes, those with the largest known

11 minimum distance for a given block length and dimension. They looked at toric codes which are determined by a polygon P and a field of size q where P fits inside a square whose sides are of

length q − 2 after an affine unimodular transformation. Equivalently, this requires that ls(P) ≤ 2 q−2 where  denotes the unit square in R . They implemented a number of algorithms, including one to compute the lattice size with respect to the unit square, to find a new champion code. Their lattice size algorithm is not particularly fast. They argue that if an affine unimodular transformation is to yield a “smaller” polytope, then its rows, when treated as vectors, must not have a norm that is too large. This allows them to consider all possible unimodular transformations that can arise from rows with norm less than some specified value. In fact, this idea can be extended to give an algorithm—albeit a slow one—in many other situations.

3.3 Brute-force algorithms

We extend Brown’s and Kasprzyk’s algorithm [5] to compute the lattice size with respect to the unit cube. This algorithm is not quick but has greater applicability than the other algorithms mentioned.

Suppose that P ⊂ l, but P 6⊂ (l − 1). If there exists an A ∈ GL(n,Z) such that AP + u ⊂ n (l − 1) for some u ∈ Z , then the lattice width of P in the direction of each row vector of A is at most l − 1 since wei (AP) ≤ l − 1 for standard basis vectors ei. We also have

T T wei (AP) = maxhei,Axi − minhei,(Ax)i = maxhA ei,xi − minhA ei,xi = wAT e (P). x∈P x∈P x∈P x∈P i

Let M be the center of mass of P and let R be the radius of the largest circle C centered at M that

l−1 fits inside P. We shift P so that the origin is at M. If ||v|| > 2R then

wv(P) ≥ wv(C) = 2||v||R > l − 1.

Hence if we want to find an A ∈ GL(n,Z) such that AP ⊂ (l − 1) we only need to consider l−1 lattice vectors v with ||v|| ≤ 2R and check if we can find n of them that can be used as rows to form

12 a unimodular matrix A. The algorithm would then search through all possible size n collections

n l−1 of primitive lattice vectors in Z with norm at most 2R and check if such a collection forms a parallelepiped of volume 1. The output is a unimodular matrix A such that AP fits inside the

smallest possible dilate of  after a translation, i.e. this matrix computes the lattice size of P with respect to . Since the n-dimensional simplex is contained in [0,1]n for every n, the same algorithm works for the lattice size with respect to the simplex if we consider containment inside dilates of the simplex instead of the cube.

3.4 The Onion Skins Algorithm

We now discuss the onion skins algorithm in more detail. Castryck and Cools proved that this algorithm is optimal in the category of toric varieties and also showed it applied to related problems.

( ) n First, we define an operation on P. Let P 1 = conv(int(P) ∩ Z ) denote the convex hull of the interior lattice points of P (the notation belies the mouthful it represents). The main result of Castryck’s and Cools’ paper is the following:

 (1) Theorem 3.4.1. If P is a lattice polygon, then ls∆(P) = ls∆ P + 3 except for a short list of exceptional polygons.

 (1) (1) (1) It is not hard to see that we must have ls∆(P) ≤ ls∆ P + 3. If P ⊂ l∆, then P ⊂ (l∆) = (l − 3)∆ (see Figure 4).

Figure 4: Convex hull of interior lattice points

13 Working out the non-equality cases, however, is significant work and constitutes much of their paper. This theorem gives the following algorithm:

Algorithm The Onion Skins Algorithm function ONIONSKINS(P) if P is exceptional then return the lattice size of the exceptional P else return ONIONSKINS(P(1))+3

In each iteration, the interior lattice points of P must be determined so that P(1) can be calculated. This is a rather expensive operation for large polygons. One of the outcomes of this dissertation is an algorithm that avoids enumerating interior lattice points. Castryck and Cools also prove that this algorithm can be adapted to determine the lattice size with respect to the unit square.

14 Chapter 4

Computing the Lattice Size with respect to the Cube

4.1 Definitions

n n n n Definition Let P ⊂ R be a lattice polytope and let  = [0,1] ⊂ R denote the unit cube in R .

Then ls(P), the lattice size of P with respect to the unit cube, is the smallest l such that P is contained in the l-dilate l of the unit cube after an affine unimodular transformation T.

Let e(P) be the smallest l such that P is contained in l after a lattice translation. Then

ls(P) = mine(AP) where the minimum is taken over all unimodular matrices A ∈ GL(n,Z).

2 Example Let P ⊂ R be the triangle with vertices (0,0), (1,0), and (2,3). Then e(P) = 3 while ls(P) = 2.

Figure 5: Lattice size with respect to the unit square

To get from the first polygon to the second we apply the map

       x 1 0 x 0          7→    +  . y −1 1 y 1

For a linear function f (x) = f (x1,...,xn) = a1x1 + ··· + anxn with a1,...,an ∈ Z and a lattice

15 n polytope P ⊂ R denote

δP( f ) = max f (x) − min f (x). x∈P x∈P

Note that δP( f ) is the lattice width wa(P) of P in the lattice direction a = (a1,...,an). When

this does not create a confusion, we will write δ( f ) instead of δP( f ). Clearly, δ( f ± g) ≤ δ( f ) +

δ(g), δ(a f ) = |a|δ( f ), and e(P) = max{δ(x1),...,δ(xn)}. Also, for an n × n matrix A = (ai j) we have

e (AP) = max {δ(ai1x1 + ··· + ainxn)} = max {wai (P)},  i=1,...,n i=1,...,n where ai = (ai1,...,ain) is the ith row of A.

4.2 Lattice size of polygons with respect to the unit square

In this section we explain a very fast algorithm for computing lattice size of lattice polygons

P ⊂ R2 based on the following theorem.

Theorem 4.2.1. Suppose that e(P) = l and δP(x ± y) ≥ l. Then δ(ax + by) ≥ l for all primitive 2 directions (a,b) ∈ Z , except, possibly, for (a,b) = (±1,0) or (0,±1). This implies that ls(P) = l and w(P) = min{δP(x),δP(y)}.

Proof. We need to check that there is no unimodular matrix A such that e(AP) < l. If such a matrix existed then the lattice width of P in the direction of each of its row vectors would be less than l, so it is enough to check that for all primitive lattice vectors (a,b) 6= (±1,0) or (0,±1) we have δ(ax + by) ≥ l. Suppose first that a and b have same sign, that is, sgn(a)·sgn(b) ≥ 0. If we also have |a| ≥ |b|, then |a − b| = |a| − |b| and we get

|a|l ≤ δ(ax + ay) ≤ δ(ax + by) + δ(ay − by) ≤ δ(ax + by) + |a − b|l.

This implies that δ(ax+by) ≥ l(|a|−|a−b|) = l|b| ≥ l, provided that b 6= 0. Similarly, if |a| ≤ |b|

16 we use

|b|l ≤ δ(bx + by) ≤ δ(ax + by) + δ(−ax + bx) ≤ δ(ax + by) + |a − b|l,

to get δ(ax + by) ≥ l(|b| − |a − b|) = l|a| ≥ l, provided that a 6= 0. Next, suppose that a and b have opposite signs, that is, sgn(a) · sgn(b) ≤ 0. If |a| ≥ |b| then |a + b| = |a| − |b| and we get

|a|l ≤ δ(ax − ay) ≤ δ(ax + by) + δ(−ay − by) ≤ δ(ax + by) + |a + b|l,

which implies δ(ax+by) ≥ l(|a|−|a+b|) = l|b| ≥ l, if b 6= 0. Similarly, if |a| ≤ |b| then |a+b| = |b| − |a| and

|b|l ≤ δ(bx − by) ≤ δ(−ax − by) + δ(ax + bx) ≤ δ(ax + by) + |a + b|l.

Hence δ(ax + by) ≥ l(|b| − |a + b|) = l|a| ≥ l, if a 6= 0. Finally, if either a or b is zero then the primitive direction is one of (±1,0), (0,±1).

We next prove an interesting observation that if a lattice polygon P ⊂ l touches all four sides

of the square l then ls(P) = l.

Theorem 4.2.2. Suppose that δP(x) = δP(y) = l. Then ls(P) = l and

w(P) = min{l,δP(x + y),δP(x − y)}.

Proof. Consider a primitive direction (a,b) ∈ Z2. We have

|a|l = δ(ax) ≤ δ(ax + by) + δ(by) = δ(ax + by) + |b|l,

17 which implies that δ(ax + by) ≥ (|a| − |b|)l ≥ l, provided that |a| > |b|. Similarly, we have

|b|l = δ(by) ≤ δ(ax + by) + δ(ax) = δ(ax + by) + |a|l, which gives δ(ax + by) ≥ l if |a| < |b|. If |a| = |b| then the primitive direction (a,b) is one of (±1,1),(1,±1). Hence we may only have lattice width smaller than l in the directions (1,1) or

(1,−1), so w(P) = min{l,δ(x + y),δ(x − y)}. Since these two directions do not give rise to a unimodular matrix, we get ls(P) = l.

Theorem 4.2.3. There is an algorithm for finding ls(P) of a lattice polygon P, which is linear in e(P) and does not require enumeration of lattice points in P.

Proof. Note that if we are in a situation described either in Theorem 4.2.1 or Theorem 4.2.2 we have already found both ls (P) and w(P).      1 0 0 1 If not, we use one of A =  ,   to pass to A(P) with strictly smaller e (P)      1 ±1 1 ±1 and repeat the process. This algorithm will terminate in at most e(P) − 1 steps. The product of such A’s at all the steps will be the unimodular map that maps P into the smallest possible lattice square.

Our next goal is to unimodularly map P into the smallest axis-parallel rectangle. Recall that

2 the product order on N is described by defining (a1,a2) ≤ (b1,b2) iff a1 ≤ b1 and a2 ≤ b2. Given 2 2 a lattice polygon P, let S1,1 be the set of pairs (a,b) ∈ N such that the rectangle [0,a]×[0,b] ⊂ R contains a unimodular copy of P. We now recover a theorem proved in [6].

Theorem 4.2.4. The set S1,1 admits a minimum with respect to the product order at the pair

(w(P),ls(P)).

Proof. We only need to show that P can be mapped uimodularly inside the rectangle R := [0,w(P)]×

[0,ls(P)]. After we run the algorithm of Theorem 4.2.3 we end up either in a situation described in the assumptions of Theorem 4.2.1, so P is unimodularly mapped inside R, or we are under

18 the assumptions of Theorem 4.2.2, where δ(x) = δ(y) = ls(P) and w(P) = min{ls(P),δ(x + y),δ(x − y)}. In the latter case, if w(P) = ls (P), we have already mapped P inside R. Otherwise,    1 ±1   A =   will finish the task. 0 1

4.3 Lattice size in dimension 3

We now work on developing an algorithm for computing the lattice size of a lattice polytope P

3 in R . Let δP(x) = l1 ≤ δP(y) = l2 ≤ δP(z) = l. Our goal now is to show that if in some directions the lattice width of P is at least l, the same is true for almost all other directions.

Theorem 4.3.1. Suppose that δP(mx + ny + z) ≥ l for all m,n ∈ Z. Then δP(ax + by + cz) ≥ l for all primitive directions (a,b,c) with nonzero c, except, possibly, for directions (a,b,c) that satisfy

|a| = |b| = 1 and |c| = 2.

Proof. Fix a primitive direction (a,b,c) ∈ Z3 with nonzero c. Let mc be the multiple of c nearest to a and nc be the multiple of c nearest to b. Consider the computation:

|c|l ≤ δ(cmx + cny + cz)

≤ δ(ax + by + cz) + δ((mc − a)x + (nc − b)y)

≤ δ(ax + by + cz) + |mc − a|l + |nc − b|l

and its consequence:

δ(ax + by + cz) ≥ |c|l − |mc − a|l − |nc − b|l. (4.1)

Suppose c is odd. By our choice of m and n we have

|c| − 1 |c| − 1 |mc − a| ≤ and |nc − b| ≤ . 2 2

19 With (4.1), this gives

|c| − 1 |c| − 1 δ(ax + by + cz) ≥ |c|l − l − l = l. 2 2

Next let c be even. Then |c| |c| |mc − a| ≤ and |nc − b| ≤ . 2 2

Notice that if c 6= 2, we cannot have equality in both inequalities. Indeed, if this were the case, for c = 2k, we would have that k divides gcd(a,b,c). Thus, either |mc − a| ≤ (|c| − 2)/2 or |nc − b| ≤ (|c| − 2)/2 and, using this in (4.1), we get

|c| |c| − 2 δ(ax + by + cz) ≥ |c|l − l − l = l. 2 2

This leaves only the case when |c| = 2. If either a or b is even then one of the differences

|mc − a|, |nc − b| is zero and the other is zero or one. Then it follows that δ(ax + by + cz) ≥ l. For the remainder of the proof, suppose both a and b are odd and |c| = 2. If δ(x + y) < l, then

δ(ax + by + cz) + l ≥ δ(ax + by + cz) + δ(x + y)

≥ δ ((a + 1)x + (b + 1)y + cz) a + 1 b + 1  ≥ 2δ x + y + z c c

≥ 2l

where the final inequality holds since we assumed δP(mx + ny + z) ≥ l for all m,n ∈ Z. It follows that δ(ax + by + cz) ≥ l. Similarly, we get the same result if δ(x − y) < l. Now, let δ(x ± y) ≥ l and suppose that neither |a| nor |b| is equal to one. Since both of them are odd, each of their absolute values is at least 3. Assume that |a| ≥ |b| and sgn(a) · sgn(b) ≥ 0. We have

δ(ax + by + cz) + δ((a − b)y − cz) ≥ δ(ax + ay) ≥ |a|l.

20 Since δ((a − b)y − cz) ≤ |a − b|l + |c|l = (|a| − |b| + |c|)l, we get

δ(ax + by + cz) ≥ |a|l − (|a| − |b| + |c|)l = (|b| − |c|)l.

Because |b| > |c| = 2, the result holds. Using the inequality δ(x − y) ≥ l, we cover the case when sgn(a) = −sgn(b). The case when |b| ≥ |a| > 1 is also covered by switching the roles of a and b in the argument above. We are left with the case when |c| = 2 and at least one of |a|, |b| is equal to one. If |b| = 1 and |a| > 1, then |a| ≥ 3. We can assume that c = 2, passing, if necessary, from (a,b,c) to (−a,−b,−c), and get

δ(ax + by + 2z) + δ(−by + (|a| − 2)z) ≥ δ(ax + |a|z) = |a|δ(sgn(a)x + z) ≥ |a|l.

We then conclude that δ(ax + by + 2z) ≥ (|a| − 1 − (|a| − 2))l = l. The argument for |a| = 1 and |b| > 1 is analogous and leaves only the case with |c| = 2 and |a| = |b| = 1.

2 Our next goal is to show that one can ensure δP(ax+by+z) ≥ l for all (a,b) ∈ Z by checking that this inequality holds true for a finite number of such pairs (a,b).

Lemma 4.3.2. Let δP(x) = l1 ≤ δP(y) = l2 ≤ δP(z) = l. Suppose that δP(x±y) ≥ l2. If for a,b ∈ Z

we have δP(ax + by + z) < l then

(1) If |a| ≥ |b| then |b| ≤ 2l−1 ; l2

(2) If |b| ≥ |a| then |a| ≤ 2l−1 . l1

Proof. Suppose sgn(a) · sgn(b) ≥ 0 and |a| ≥ |b|. We have

|a|δ (x + y) ≤ δ (ax + by + z) + δ ((a − b)y − z),

21 which implies

|a|l2 < l + δ ((a − b)y − z) ≤ l + |a − b|δ(y) + δ(z) ≤ 2l + (|a| − |b|)l2.

Rearranging the final inequality gives |a|l2 − (|a| − |b|)l2 < 2l which lets us conclude that |b| ≤

(2l − 1)/l2. If |b| ≥ |a|, the computation is similar and we conclude that |a| ≤ (2l − 1)/l1. If sgn(a) = −sgn(b) and |a| ≥ |b|, then we use the remaining part of our hypothesis:

|a|l2 ≤ δ(ax − ay) ≤ δ(ax + by + z) + δ((a + b)y + z) < 2l + (|a| − |b|)l2

which again implies |a| ≤ (2l −1)/l1. The case sgn(a) = −sgn(b) and |b| ≥ |a| is treated similarly.

Lemma 4.3.3. Let δ(x) = l1 ≤ δ(y) = l2 ≤ δ(z) = l. If δ(ax + by + z) < l for a,b ∈ Z then

2l − 1 + |b|l 2l − 1 + |a|l |a| ≤ 2 and |b| ≤ 1 . l1 l2

Proof. Consider

|a|l1 = δ(ax) ≤ δ(ax + by + z) + δ(by) + δ(z) ≤ 2l − 1 + |b|l2.

Dividing by l1 we get 2l − 1 + |b|l |a| ≤ 2 . l1

Starting from |b|l2 yields the other bound.

Definition Let set S consist of all pairs (a,b) ∈ Z2 that satisfy

1. If |a| ≥ |b| then |b| ≤ 2l−1 and |a| ≤ 2l−1+|b|l2 ; l2 l1

2. If |b| ≥ |a| then |a| ≤ 2l−1 and |b| ≤ 2l−1+|a|l1 . l1 l2

2 Proposition 4.3.4. The set S is finite and its size does not exceed 64l . l1l2

22 Proof. If (a,b) ∈ S then if |a| ≤ |b| we have |a| < 2l and |b| < 4l and, similarly, if |b| ≤ |a|, we l1 l2 have |b| < 2l and |a| < 4l , which demonstrates that S is finite and the bound of the proposition l2 l1 follows.

This shows that there are at most finitely many pairs (a,b) that may satisfy δ(ax + by + z) < l under the assumptions of Lemmas 4.3.2 and 4.3.3. We next consider a particular case when set S is very small. The proof of this proposition is a direct application of Lemmas 4.3.2 and 4.3.3.

Proposition 4.3.5. Let δP(x) = δP(y) = δP(z) = l and suppose that δP(x ± y) ≥ l. Then we may have δP(ax + by + z) < l only for pairs (a,b), where (|a|,|b|) is in the set {(1,1),(1,2),(2,1)}.

We now formulate our main result.

Theorem 4.3.6. Let δP(x) = l1 ≤ δP(y) = l2 ≤ δP(z) = l. Suppose that δP(x±y) ≥ l2 and δP(ax+ by + z) ≥ l for all (a,b) ∈ S. Then ls(P) = l.

Proof. By Lemmas 4.3.2 and 4.3.3, we have that δ(mx+ny+z) ≥ l for all (m,n) ∈ Z2. Applying Theorem 4.3.1, we know that δ(ax + by + cz) ≥ l for all primitive (a,b,c) ∈ Z3 except, possibly, for triples (a,b,c) with |c| = 0 or 2. If there existed a unimodular matrix A with e(AP) < l then the width of P in the direction of each of its rows would have been less than l. Hence the rows of A would have to have last components equal to 0 or 2, but then its determinant would be even.

Theorem 4.3.6 leads to an algorithm for computing lattice size ls(P) for a lattice polytope 3 P ⊂ R . Without loss of generality, switching x,y, and z, we can assume that δP(x) ≤ δP(y) ≤ δP(z). Applying the 2D algorithm of Theorem 4.2.3 to the projection of P to the (x,y)-plane we can ensure that δP(x ± y) ≥ l2, where δP(x) = l1 ≤ δP(y) = l2 ≤ δP(z) = l. According to Theorem 4.2.1 this 2 implies that δ(ax + by) ≥ l2 for all primitive (a,b) ∈ Z except, possibly, (a,b) = (±1,0).

Our next step is to check whether for all directions (a,b) in the set S we have δP(ax+by+z) ≥ l.

If this is the case then by Theorem 4.3.6 we conclude that ls(P) = l. Otherwise, we find (a,b) ∈ S

23 0 such that δP(ax + by + z) < l and repeat the process with P replaced with P = AP, where

  1 0 0     A = 0 1 0.     a b 1

Since δP0 (x) + δP0 (y) + δP0 (z) < l1 + l2 + l, the algorithm will terminate after finitely many steps, that is, we will arrive at a situation where the assumptions of Theorem 4.3.6 are satisfied. Since by Proposition 4.3.4 each step takes O(l2) operations and we have O(l) steps, this algo-

3 3 rithm will take O(l ) operations, or using our notation from before, O(e(P) ) operations. A more careful analysis of the algorithm will help us bring this bound down. Let’s first assume

that l2 < l. As before, we use the 2D algorithm to ensure that δP(x ± y) ≥ l2. Note that each step

in the 2D algorithm takes at most 4 checks and reduces l1 + l2 + l by at least 1. Next we search through S to find a direction (m,n,1) ∈ S that corresponds to the smallest possible lattice width l0 of P over all the directions in S and let z0 = mx + ny + z. We can assume that l0 < l for otherwise the algorithm will terminate at this step.

0 0 0 If l ≥ l2 then δ(x) = l1 ≤ δ(y) = l2 ≤ δ(z ) = l and δ(x + y) ≥ l2. Now pick an arbitrary

(a,b) ∈ Z2. If (a + m,b + n,1) ∈ S then

δ(ax + by + z0) = δ((a + m)x + (b + n)y + z) ≥ l0.

Otherwise,

δ(ax + by + z0) = δ((a + m)x + (b + n)y + z) ≥ l > l0,

0 0 and using Theorem 4.3.6 we conclude that in the case l ≥ l2 we have ls(P) = l . Hence we only need to go through S once to either terminate the algorithm or to reduce l by at

 l2   l2  least l − l2. That is, we are using O operations to make l − l2 steps, which is O l1l2 l1l2(l−l2) operations per step.

Note that l2(l − l2) ≥ l − 1 since l2(l − l2) is an upside down parabola when considered as a

24 function in l2 and since 1 ≤ l2 ≤ l − 1 the smallest value occurs at l2 = l − 1 and l2 = 1. Hence we 2 2 2 get l ≤ l ≤ l = O(l). l1l2(l−l2) l1(l−1) l−1 2 If l = l then S is of size at most 64l ≤ 64l = O(l). We conclude that each reduction of l +l +l 2 l1l 1 2

by 1 requires O(l) operations and hence the algorithm is quadratic in e(P). We have proved the result formulated in the theorem below.

Theorem 4.3.7. There exists an algorithm for finding the lattice size of a 3D lattice polytope P,

2 which takes O(e(P) ) operations and does not require enumeration of lattice points in P.

Corollary 4.3.8. The algorithm of Theorem 4.3.7 can be used to find the lattice width of P.

Proof. After we run this algorithm we end up in a situation where

δ(x) = l1 ≤ δ(y) = l2 ≤ δ(z) = l = ls(P),

δ(x ± y) ≥ l2, and

δ(ax + by + z) ≥ l

2 for all (a,b) ∈ Z . By Theorem 4.2.1 this implies that δ(ax + by) ≥ l2 for all primitive directions (a,b), except, possibly, for (a,b) = (±1,0) or (0,±1). Also, by Theorem 4.3.6 we have that

∆(ax + by + cz) ≥ l for all primitive directions (a,b,c) with c 6= 0, except, possibly, for the case when |a| = |b| = 1 and |c| = 2. We denote

3 E = {(a,b,c) ∈ Z | |a| = |b| = 1 and |c| = 2}.

Now to find w(P) one needs to find the minimum of the lattice width of P with respect to all directions in E and then pick the smaller one of this minimum and l1.

Based on our 2D results, one may hope that in the 3D case the set of directions S that one needs to check would consist of directions whose components have absolute value of at most 1. The example below demonstrates that the conditions δP(x±y±z) ≥ l, δP(x±z) ≥ l, δP(y±z) ≥ l, and

δP(x ± y) ≥ l2 do not necessarily imply that ls(P) = l.

25 Example Let P be the convex hull of the following set of points in R3

{(1,0,0),(1,1,0),(0,4,0),(2,4,0),(0,1,1),(0,4,1),(0,1,10)}.

Then δ(x) = 2, δ(y) = 4, δ(z) = 10, δ(x + y + z) = 10, δ(x − y + z) = 13, δ(x + y − z) = 15, δ(x − y − z) = 12, δ(x + z) = 10, δ(x − z) = 12, δ(y + z) = 11, δ(y − z) = 13, δ(x + y) = 5,   1 0 0     δ(x − y) = 5, but δ(2x + y + z) = 9, so one can apply A = 0 1 0 to get e (AP) = 9.      2 1 1

3 Definition Let P ⊂ R be a lattice polytope. We define w2(P) to be the lattice size with respect to the set [0,1] × [0,1] × R.

Now we observe that it is impossible for P to have lattice width less than l2 with respect to two directions in E. Indeed, if (a1,b1,c1) and (a2,b2,c2) are two such distinct directions then all the components of the sum of these two directions are even, and hence we get

a + a b + b c + c  δ(a x + b y + c z) + δ(a x + b y + c z) ≥ 2δ 1 2 x + 1 2 y + 1 2 z ≥ 2l , 1 1 1 2 2 2 2 2 2 2 where, for the last inequality, we observe that half-sum of two distinct directions in E cannot be in E.

Hence to compute w2(P), we need to find the minimum m of the lattice width of P with respect to all the directions in E and then we pick the smallest two numbers out of m, l1, and l2. If the two smallest ones are l1 and l2 then w2(P) = l2. If we have l1 ≤ m < l2 then w2(P) = m since we can   1 0 0     apply to P a unimodular map A = a b c, where (a,b,c) is a direction with |a| = |b| = 1 and     0 0 1 |c| = 2 such that the lattice width of P in that direction equals m.

In the case m < l1 ≤ l2 we would have w2(P) = l1 and we would use the same map A as before, but with first two rows switched. We have proved:

26 Theorem 4.3.9. Let P ⊂ R3 be a lattice polytope. Then there exists an algorithm for finding w(P), w2(P), and ls(P) which is quadratic in e(P) and does not require enumeration of lattice points in P.

The product order on N3 is described by defining

(a1,b1,c1) ≤ (a2,b2,c2) iff a1 ≤ a2, b1 ≤ b2, and c1 ≤ c2.

3 3 Given a lattice polytope P ⊂ R , let S1,1,1 be the set of triples (a,b,c) ∈ Z such that the rectangle [0,a] × [0,b] × [0,c] ⊂ R3 contains a unimodular copy of P.

Theorem 4.3.10. The set S1,1,1 admits a minimum with respect to the product order at the triple

(w(P),w2(P),ls(P)).

Proof. We only need to show that P can be mapped unimodularly inside R = [0,w(P)]×[0,w2(P)]×

[0,ls(P)]). If after we run the algorithm we end up in the situation where l1 ≤ l2 ≤ m then P is already inside R. If we get l1 ≤ m < l2, then the map A as above will map P inside R. Finally, if m ≤ l1, then this map A with first two rows switched will finish the task.

Example In the view of Theorem 4.2.2 it is natural to ask whether for a lattice polytope P ⊂ R3

with δP(x) = δP(y) = δP(z) = l we can conclude that ls(P) = l. The answer is negative, as demonstrated by the following example. Let P be the convex hull of the set

{(0,3,1),(5,2,3),(4,0,4),(2,5,4),(1,3,0),(3,4,5)},

    1 1 0 −3         so we have δP(x) = δP(y) = δP(z) = 5. If we apply A = 1 0 −1 to this set and shift by  2          1 1 −1 0 the image is the convex hull of

{(0,1,2),(4,4,4),(1,2,0),(4,0,3),(1,3,4),(4,0,2)},

27 for which δ(x) = δ(y) = δ(z) = 4.

28 Chapter 5

Computing the Lattice Size with respect to the Simplex

In this chapter, we discuss a new algorithm to determine the lattice size of a polygon with respect to the standard simplex that does not require the enumeration of interior lattice points. This is explained in Theorem 5.2.8 which is based on Theorem 5.2.7.

5.1 Definitions

Let ∆ be the standard simplex in the plane with the vertices (0,0),(1,0),(0,1). We define the lattice size ls∆(P) of a lattice polygon P with respect to the standard simplex to be the smallest non-negative integer l such that P fits into l∆, the l-dilate of the standard simplex, after an affine unimodular transformation. We define the naive lattice size nls(P) to be the smallest non-negative integer l such that a lattice polygon P is contained in l∆ after a lattice translation. Note that this definition is inherently translation-invariant. An explicit expression for nls(P) is:

nls(P) = max (x + y) − min (x) − min (y). (x,y)∈P (x,y)∈P (x,y)∈P

  a b   After we apply a transformation T =  , the expression becomes c d

nls(TP) = max ((a + c)x + (b + d)y) − min (ax + by) − min (cx + dy). (5.1) (x,y)∈P (x,y)∈P (x,y)∈P

29 For any S ⊂ GL(2,Z), let nlsS (P) = min{nls(TP) : T ∈ S }. We will often refer to the matrices

          1 0 1 1 0 −1 0 1 −1 −1           L =  , U =  , R =  , F =  , and S =  . 1 1 0 1 1 0 1 0 1 0

The matrices L and U skew the plane vertically and horizontally, respectively. The matrix R is

◦ a 90 counter-clockwise rotation of the plane about the origin. If ϕS(x,y) = S(x,y) + (1,0) and

ϕF (x,y) = (y,x), then the group generated by these elements hϕS,ϕF i ≤ AGL(2,Z) is exactly the group of affine transformations that map ∆ to itself.

Proposition 5.1.1. The group of affine transformations that maps l∆ to itself is hϕS,ϕF i for any l ∈ N.

∼ Proof. It is easy to verify that f (l∆) = l∆ for each f ∈ hϕS,ϕF i and that hϕS,ϕF i = S3, the sym- metric group on 3 symbols. Since each affine transformation that preservers a polytope necessarily maps vertices to vertices, the group in the statement can have no more than |S3| = 6 elements.

Thus, hϕS,ϕF i is the full group of affine unimodular transformations that preserves l∆.

It follows immediately from Proposition 5.1.1 and the translation invariance of nls that

nlshS,Fi(P) = nls(P) for any lattice polygon P. This symmetry will play an important role in the proof of our main theorem.

5.2 Main Results

In the remainder of this chapter, we will suppose that nls(P) = l and P ⊂ l∆. The last condition can always be satisfied by translating P if necessary and allows us to define regions of l∆ that will be useful for obtaining bounds. Furthermore, this assumption means that max(x,y)∈P(x + y) = l and min(x,y)∈P(x) = min(x,y)∈P(y) = 0. Let s = min(x,y)∈P(x + y), u = max(x,y)∈P(x), and t = max(x,y)∈P(y), and define the following convex regions:

30 xu

yt

x+ys

Figure 6: Examples of s, u, and t

n u  o λ = conv{(s,0),(0,s)}, Σ = conv (0,0), ,0 ,(0,u) , 1 1 2 l + s   λ = conv{(u,0),(u,l − u)}, Σ = conv ,0 ,(l,0),(s,l − s) , 2 2 2 n u u o λ = conv{(0,t),(l −t,t)}, Σ = conv (0,l − u), ,l − ,(0,l) , 3 3 2 2 n  t t o Σ = conv (l −t,0),(l,0), l − , . 4 2 2

By definition of s, u, and t, the regions λ1 ∩ P, λ2 ∩ P, and λ3 ∩ P are non-empty, respectively. We also have the following results:

Lemma 5.2.1. If nls(−LP) ≥ l, then Σ1 ∩ P is non-empty. If nls(LP) ≥ l then Σ2 ∩ P is non-empty.   −1 0   Proof. Let us first consider the effect of −L =  . By our hypotheses, we know that −1 −1

l ≤ nls(−LP) = max (−2x − y) − min (−x) − min (−x − y) (x,y)∈P (x,y)∈P (x,y)∈P

= max (x) + max (x + y) − min (2x + y). (x,y)∈P (x,y)∈P (x,y)∈P

31 λ3

λ1 λ2

Σ3

Σ 1 Σ2 Σ4

Figure 7: Example of line segments and simplexes

By definition, max(x,y)∈P(x) = u and, because nls(P) = l and P ⊂ l∆, we have that max(x,y)∈P(x +

y) = l. Then u ≥ min(x,y)∈P(2x +y) and there must be a point of P on or below the line 2x +y = u. This point of P is in the region given by x ≥ 0, y ≥ 0, and 2x + y ≤ u or, equivalently, is a simplex whose vertices are (0,0), (u/2,0), and (0,u). Thus, Σ1 ∩ P is non-empty.

Showing that Σ2 ∩ P is non-empty follows similarly. We have

l ≤ nls(LP) = max (2x + y) − min (x) − min (x + y) (x,y)∈P (x,y)∈P (x,y)∈P

= max (2x + y) − s. (x,y)∈P

Thus, there is a point of P contained in the region defined by 2x + y ≥ l + s, x ≥ 0, and x + y ≤ l.

This region is precisely Σ2 and so Σ2 ∩ P is non-empty.

32 Lemma 5.2.2. If nls(−RP) ≥ l, then Σ3 ∩P is non-empty. If nls(RP) ≥ l then Σ4 ∩P is non-empty.

Proof. We begin by showing Σ ∩ P is non-empty. For this, we consider the effect of −R =   3 0 1     on the naive lattice size of P: −1 0

l ≤ nls(−RP) = max (y − x) − min (y) − min (−x) (x,y)∈P (x,y)∈P (x,y)∈P

= max (y − x) + u. (x,y)∈P

With this computation, our assumptions imply max(x,y)∈P(y − x) ≥ l − u. Since P is contained in l∆, it must have a point in the polygon whose halfspace description is y − x ≥ l − u, x ≥ 0, and x + y ≤ l. This region is Σ3.

To show that Σ4 ∩ P is nonempty, we look at the effect of R:

l ≤ nls(RP) = max (x − y) − min (−y) − min (x) = max (x − y) +t. (x,y)∈P (x,y)∈P (x,y)∈P (x,y)∈P

We assumed nls(RP) ≥ l so it follows that max(x,y)∈P(x − y) ≥ l −t. Thus, there is a point of P in the polygon given by x − y ≥ l −t, y ≥ 0, and x + y ≤ l. As we wanted, this polygon is Σ4.

Using the fact that these regions are non-empty, we prove a couple lemmas that will form the foundation for our main theorem.

Lemma 5.2.3. Suppose nls{±L}(P) ≥ l. If α − β ≥ 2 and β ≥ 2, then w(α,β)(P) ≥ l.

Proof. Our strategy will be to pick appropriate points of P and then use the fact that, over a convex polytope, a linear function obtains its maximum and minimum at a vertex. This will allow us to

bound w(α,β)(P) from below. There are 4 cases based on the relationship between α and β and

the relationship between s and l. Note that Lemma 5.2.1 ensures that we have points in Σ1 ∩ P and

Σ2 ∩ P.

To begin, we assume that α/2 ≥ β. If s ≥ l/2, take (x1,y1) ∈ Σ1 ∩ P. Since Σ1 is a convex  u polytope, it follows that αx1 + βy1 ≤ max 0,α 2 ,βu . Given our assumptions on α and β, we

33 must have αx1 + βy1 ≤ (α/2)u. We choose another point (x2,y2) ∈ λ2 ∩ P. Then αx2 + βy2 ≥ min{αu,αu + β(l − u)} = αu since β ≥ 0. Now we bound the lattice width:

w(α,β)(P) = max (αx + βy) − min (αx + βy) (x,y)∈P (x,y)∈P

≥ αx2 + βy2 − (αx1 + βy1) α ≥ αu − u ≥ 2u ≥ 2s ≥ l. 2

The penultimate inequality follows because min(x,y)∈P(y) = 0. This completes the first subcase. If

s ≤ l/2, then we pick (x1,y1) ∈ λ1 ∩ P and (x2,y2) ∈ Σ2 ∩ P. Using the convexity of λ1 and Σ2, we determine that αx1 +βy1 ≤ max{αs,βs} and αx2 +βy2 ≥ min{α(l +s)/2,αl,αs+β(l −s)}.

Since α ≥ β and α/2 ≥ β, we have αx1 + βy1 ≤ αs and αx2 + βy2 ≥ αs + β(l − s), respectively. Therefore: l w (P) ≥ αs + β(l − s) − αs ≥ β ≥ l. (α,β) 2

Now we suppose that α/2 ≤ β. We again have two cases. If s ≥ l/2, pick (x1,y1) ∈ Σ1 ∩P and

(x2,y2) ∈ λ2 ∩P. With our new assumption, the bounds become αx1 +βy1 ≤ max{0,αu/2,βu} =

βu and αx2 + βy2 ≥ min{αu,αu + β(l − u)} = αu. Then the lattice width in the direction (α,β) is at least l:

w(α,β)(P) ≥ αu − βu ≥ 2u ≥ 2s ≥ l.

Finally, assume s ≤ l/2. Let (x1,y1) ∈ λ1 ∩ P and (x2,y2) ∈ Σ2 ∩ P. Here, we obtain the bounds

αx1 +βy1 ≤ max{αs,βs} = αs and αx2 +βy2 ≥ min{α(l +s)/2,αl,αs+β(l −s)} = α(l +s)/2. This gives us:

l + s l w (P) ≥ αx + βy − αx − βy ≥ α − αs ≥ α ≥ l (α,β) 2 2 1 1 2 4

since α ≥ β + 2 ≥ 4.

34   1 0   Lemma 5.2.4. Suppose that nls(−P) ≥ l and nls{±L,±R}(P) ≥ l. If T =   for some c ∈ Z, c 1 then nls(±TP) ≥ l.

Proof. The expressions we need to consider are

nls(TP) = max ((c + 1)x + y) − min (cx + y) and (x,y)∈P (x,y)∈P

nls(−TP) = max (cx + y) − min ((c + 1)x + y) + u. (x,y)∈P (x,y)∈P

The c = 0 and c = 1 cases are handled by our assumptions so we begin by considering the c ≥ 2 case. Pick (x1,y1) ∈ λ1 ∩ P and (x2,y2) ∈ Σ2 ∩ P to get the following inequalities:

cx1 + y1 ≤ max{cs,s} = cs,

(c + 1)x1 + y1 ≤ max{(c + 1)s,s} = (c + 1)s,  l + s  cx + y ≥ min c ,cl,(c − 1)s + l = (c − 1)s + l, and 2 2 2  l + s  (c + 1)x + y ≥ min (c + 1) ,(c + 1)l,cs + l = cs + l. 2 2 2

Our bounds for the naive lattice size are now:

nls(TP) ≥ (c + 1)x2 + y2 − (cx1 + y1) ≥ cs + l − cs = l and

nls(−TP) ≥ cx2 + y2 − ((c + 1)x1 + y1) + u ≥ (c − 1)s + l − (c + 1)s + u

= l − 2s + u.

These show that nls(TP) ≥ l and if u ≥ 2s then nls(−TP) ≥ l . If u ≤ 2s, pick (x1,y1) ∈ Σ1 ∩ P

35 and (x2,y2) ∈ Σ2 ∩ P to get that (c + 1)x1 + y1 ≤ max{0,(c + 1)u/2,u} = (c + 1)u/2. This yields

u nls(−TP) ≥ cx + y − ((c + 1)x + y ) + u ≥ (c − 1)s + l − (c + 1) + u 2 2 1 1 2  u ≥ l + (c − 1) s − ≥ l. 2

Thus, nls(±TP) ≥ l for non-negative values of c. If c = −1 then ST = R. Since nls(STP) = nls(TP), this case is handled by our hypotheses.

Therefore, we suppose c ≤ −2. If (x1,y1) ∈ Σ4 and (x2,y2) ∈ λ3, then

n  t  t o cx + y ≤ max c(l −t),cl,c l − + = c(l −t), 1 1 2 2

(c + 1)x1 + y1 ≤ (c + 1)(l −t),

cx2 + y2 ≥ min{t,c(l −t) +t} = c(l −t) +t, and

(c + 1)x2 + y2 ≥ (c + 1)(l −t) +t.

These give the following bounds:

nls(TP) ≥ (c + 1)x2 + y2 − (cx1 + y1) ≥ (c + 1)(l −t) +t − c(l −t) = l and

nls(−TP) ≥ cx2 + y2 − ((c + 1)x1 + y1) + u ≥ c(l −t) +t − (c + 1)(l −t) + u

= t + (u +t − l).

We have shown that nls(TP) ≥ l and if u +t − l ≥ l −t then nls(−TP) ≥ l.

36 y-xl-u

yt

Figure 8: Region determined by special case

To handle the case when u +t − l ≤ l −t requires more work. In this situation, the lines y = t

and y − x = l − u intersect inside l∆. We may conlcude that there is a point (x2,y2) ∈ P contained in the region defined by y ≤ t, x ≥ 0, and y −x ≥ l −u. To see this, realize that the aforementioned

region is Σ3 ∩ {(x,y) : y ≤ t} (see Figure 8). It is a simplex with vertices at (0,l − u), (0,t), and

(u +t −l,t). By convexity, we now have cx2 +y2 ≥ min{l −u,t,c(u +t −l)+t} = c(u +t −l)+t and that

nls(−TP) ≥ c(u +t − l) +t − (c + 1)(l −t) + u

= l − (c + 1)[(l −t) − (u +t − l)] ≥ l

where we use the inequalities c ≤ −2 and u +t − l ≤ l −t. This completes the proof.

Let OL = L · h−Si = {±L,±U,±R} be the orbit of L under right multiplication by the group h−Si. We now work to show that if nls(−P) ≥ l and nlsOL (P) ≥ l, then ls∆(P) = l. Define a function π : GL(2,Z) → Z2 by

  a b   π   = (a + c,b + d). c d

37 We define the following regions in the plane:

2 σ1 = {(x,y) ∈ R : x − y ≥ 2,y ≥ 2},

2 σ2 = {(x,y) ∈ R : x − y ≥ 2,−x ≥ 2}, and

2 σ3 = {(x,y) ∈ R : −x ≥ 2,y ≥ 2}.

S The union U = i ±σi (see Figure 9) along with π will allow us to geometrically understand matrices which are easily seen to not improve the naive lattice size.

S Theorem 5.2.5. Suppose nlsOL (P) ≥ l and let T ∈ GL(2,Z). If π(T) ∈ U = i ±σi, then nls(TP) ≥ l.

Figure 9: Directions with large width

  a b   Proof. Let T =  . We have the following basic fact about minimums: c d

min ((a + c)x + (b + d)y) ≥ min (ax + by) + min (cx + dy). (x,y)∈P (x,y)∈P (x,y)∈P

38 Using equation 5.1, we get that

nls(TP) = max ((a + c)x + (b + d)y) − min (ax + by) − min (cx + dy) (x,y)∈P (x,y)∈P (x,y)∈P

≥ max ((a + c)x + (b + d)y) − min ((a + c)x + (b + d)y) (5.2) (x,y)∈P (x,y)∈P

= w(a+c,b+d)(P) = wπ(T)(P).

Suppose that π(T) ∈ σ1, i.e. (a + c) − (b + d) ≥ 2 and b + d ≥ 2. Lemma 5.2.3 lets us conclude that nls(TP) ≥ l. Since w(α,β)(P) = w(−α,−β)(P), we have the same result if π(T) ∈ −σ1. Our assumptions also give us that:

l ≤ nls{±U}(P) = nls{±L}(SP) = nls{±L}(ϕS(P))

which allows us to conclude, via (5.2), that nls(TϕS(P)) = nls(TSP) ≥ l. We have the following identity:

    b − a −a b + d − (a + c)     | π(TS) = π   =   = S π(T). d − c −c −(a + c)

Hence, if π(T) ∈ σ1, then (b + d − a − c) − (−a − c) ≥ 2 and −(b + d − a − c) ≥ 2. This shows that π(TS) ∈ σ2 when π(T) ∈ σ1. We have now proven that if nls{±U}(P) ≥ l and π(T) ∈ σ2, then nls(TP) ≥ l.

Using the same technique, we also get that if nls{±R}(P) ≥ l and π(T) ∈ ±σ3, then nls(TP) ≥ l. This completes the proof.   a b   2 Now, we consider the matrices T =   not covered by the preceding result. Since nls(S TP) = c d nls(STP) = nls(TP), π(ST) = (−c,−d), and π(S2T) = (−a,−b), we may assume that {(a,b),(c,d),(a+

c,b+d)}∩U = /0.We note that OL ·F = F ·OL which implies nlsOL (FP) = nlsOL (P). Hence, our assumptions are completely symmetric with respect to both the left- and right-multiplication action

39 of the group hS,Fi. The next result makes use of this symmetry to cover the remaining matrices.   a b Lemma 5.2.6. Suppose that nls(−P) ≥ l and nls (P) ≥ l. Let T =   ∈ GL(2, ). If OL   Z c d {(a,b),(c,d),(a + c,b + d)} ∩ U = /0, then nls(TP) ≥ l.

Proof. First, suppose that for each (x,y) ∈ {(a,b),(c,d),(a + c,b + d)} we have |x| ≤ 1, |y| ≤ 1, and |x − y| ≤ 1, i.e. each of these inequalities holds simultaneously for each (x,y). In this case, each row of T as well as its sum of rows is in the hexagon Q = conv{(1,0), (1,1), (0,1), (−1,0), (−1,−1), (−1,0)}. The unimodular matrices that can be made from these rows are exactly the elements of h−S,Fi ∪ L · h−S,Fi. By our hypotheses, none of these matrices improve l.

We now consider matrices that do not fall under the case above. There must be an (α,β) ∈ {(a,b),(c,d),(a + c,b + d)} not contained in Q that lies on one of the six lines: x = 1, −x = 1, y = 1, −y = 1, x − y = 1, and y − x = 1. We note that the group hS|,F|i acts on these lines transitively. Using this fact, we can work with a matrix T such that π(T) = (α,1) with |α| ≥ 2. That is, if we prove nls(TP) ≥ l for all such matrices using our symmetric assumptions, we can extend this to all matrices in the double coset hS,Fi · T · hS,Fi.

Let π(T) = (α,1) and suppose that |α| ≥ 2. Furthermore, since nls(FTP) = nls(TP), we will   a 1 − d   assume detT = 1. Our assumptions imply b + d = 1, so we can write T =  . If d = 1, c d   −1 0 2   then Lemma 5.2.4 lets us conclude nls(TP) ≥ l. If d = 0, then S T =   and we can −α −1 again make use of Lemma 5.2.4. Suppose now that d = −1. To satisfy our initial hypotheses, we need (a,2) 6∈ U . This forces

       −1 2 1 2 3 2        T ∈  , ,   0 −1 −1 −1 −2 −1  but none of these matrices satisfy |α| ≥ 2. Now assume that |d| ≥ 2. This yields 1 = detT = αd − c and we conclude that |1 + c| ≥ 4.

40 Equivalently, we now have that c ≥ 3 or c ≤ −5. Since |d| ≥ 2 and |c| ≥ 2, this requires us to have c = d ± 1 (see Figure 9). Hence 1 = detT = (α − 1)d ± 1. Since |α|,|d| ≥ 2, we can only have   −1 −1   c = d + 1. This forces α = d = 2 which yields T =  . Multiplying by S on the right 3 2   1 0   gives us  . This matrix does not improve the naive lattice size by Lemma 5.2.4. −2 1

Theorem 5.2.7. If nls(−P) ≥ l and nlsOL (P) ≥ l, then ls∆(P) = l.

k Proof. Let T ∈ GL(2,Z). If π(S T) ∈ U for k = 0,1,2 then nls(TP) ≥ l by Theorem 5.2.5. If this is not the case, T satisfies the hypotheses of Lemma 5.2.6 and we also conclude that nls(TP) ≥ l.

Theorem 5.2.8. There exists an algorithm for computing lattice size ls(P), where P is a lattice polygon, which does not require enumeration of lattice points in P.

Proof. This is a simple consequence of Theorem 5.2.7. For a lattice polygon P, which is rotated so

that nlshRi(P) = l, we check if nls{±L,±U}(P) ≥ l. If so, we can conclude that ls(P) = l. Otherwise we pass to the polygon TP where nls(TP) < l and repeat the previous step. To obtain a matrix which computes lattice size of P we need to multiply together matrices used at each step of this algorithm. The algorithm terminates in at most nls(P) − 1 steps.

41 Bibliography

[1] Vladimir I. Arnol’d. “Statistics of integral convex polygons”. In: Functional Analysis and

Its Applications 14.2 (1980), pp. 79–81. ISSN: 1573-8485.

[2] Imre Bar´ any´ and Janos´ Pach. “On the number of convex lattice polygons”. In: Combina- torics, Probability and Computing 1.4 (1992), 295302.

[3] Alexander Barvinok. A Course in Convexity. Graduate studies in mathematics. American

Mathematical Society, 2002. ISBN: 9780821829684.

[4] David N. Bernshtein. “The number of roots of a system of equations”. In: Functional Anal-

ysis and Its Applications 9.3 (1975), pp. 183–185. ISSN: 1573-8485.

[5] Gavin Brown and Alexander M. Kasprzyk. “Small polygons and toric codes”. In: Journal of Symbolic Computation 51 (2013). Effective Methods in Algebraic Geometry, pp. 55 –62.

ISSN: 0747-7171.

[6] Wouter Castryck and Filip Cools. “The lattice size of a lattice polygon”. In: Journal of

Combinatorial Theory, Series A 136 (2015), pp. 64 –95. ISSN: 0097-3165.

[7] Emilie´ Charrier, Fabien Feschet, and Lilian Buzer. “Computing efficiently the lattice width in any dimension”. In: Theoretical Computer Science 412.36 (2011). Advances in Discrete

Geometry for Computer Imagery, pp. 4814 –4823. ISSN: 0304-3975.

[8] Jan Draisma, Tyrrell B. McAllister, and Benjamin Nill. “Lattice-Width Directions and Minkowski’s 3d-Theorem”. In: SIAM Journal on Discrete Mathematics 26.3 (2012), pp. 1104–1107.

[9] Fabien Feschet. “The Exact Lattice Width of Planar Sets and Minimal Arithmetical Thick- ness”. In: Combinatorial Image Analysis. Ed. by Ralf Reulke et al. Berlin, Heidelberg:

Springer Berlin Heidelberg, 2006, pp. 25–33. ISBN: 978-3-540-35154-2.

42 [10] Christian Haase and Gunter¨ M. Ziegler. “On the Maximal Width of Empty Lattice Sim-

plices”. In: European Journal of Combinatorics 21.1 (2000), pp. 111 –119. ISSN: 0195- 6698.

[11] Anthony Harrison and Jenya Soprunova. “Lattice size of 2D and 3D polytopes with respect

to the unit cube”. In: ArXiv e-prints (2017). arXiv: 1709.03451 [math.CO].

[12] Anthony Harrison, Jenya Soprunova, and Patrick Tierney. “Lattice size of polygons with re-

spect to the standard simplex”. In: ArXiv e-prints (2017). arXiv: 1709.03454v2 [math.CO].

[13] Anatoli G. Kouchnirenko. “Polyedres` de Newton et nombres de Milnor”. In: Inventiones

mathematicae 32.1 (1976), pp. 1–31. ISSN: 1432-1297.

[14] Jeffrey Lagarias and Gunter¨ Ziegler. “Bounds for lattice polytopes containing a fixed num- ber of interior points in a sublattice”. In: Canadian Journal of Mathematics 43 (1991), pp. 1022 –1035.

[15] Josef Schicho. “Simplification of Surface Parametrizations”. In: Proceedings of the 2002 In- ternational Symposium on Symbolic and Algebraic Computation. ISSAC ’02. Lille, France:

ACM, 2002, pp. 229–237. ISBN: 1-58113-484-3.

[16] Josef Schicho. “Simplification of surface parametrizationsa lattice polygon approach”. In:

Journal of Symbolic Computation 36.3 (2003). ISSAC 2002, pp. 535 –554. ISSN: 0747- 7171.

[17] Gunter¨ M. Ziegler. Lectures on Polytopes. Graduate Texts in Mathematics. Springer New

York, 2012. ISBN: 9780387943657.

43