ALGORITHMS FOR COMPUTING THE LATTICE SIZE A dissertation submitted to Kent State University in partial fulfillment of the requirements for the degree of Doctor of Philosophy by Anthony Harrison August 2018 c Copyright All rights reserved Except for previously published materials Dissertation written by Anthony Harrison B.S., Texas State University–San Marcos, 2010 M.S., Texas State University–San Marcos, 2013 Ph.D., Kent State University, 2018 Approved by Jenya Soprunova , Chair, Doctoral Dissertation Committee Ivan Soprunov , Members, Doctoral Dissertation Committee Mark Lewis Mikhail Chebotar Feodor Dragan Austin Melton Accepted by Andrew Tonge , Chair, Department of Mathematical Sciences James L. Blank , Dean, College of Arts and Sciences TABLE OF CONTENTS . iii LIST OF FIGURES . iv ACKNOWLEDGMENTS . v 1 INTRODUCTION . 1 2 LATTICES AND POLYTOPES . 2 2.1 Affine and Convex Geometry . 2 2.2 Lattices . 3 2.3 Polytopes . 4 2.4 Lattice Width . 6 2.5 Newton Polytopes . 7 3 LATTICE SIZE . 9 3.1 Introduction . 9 3.2 Algebraic Geometry and the Lattice Size . 10 3.3 Brute-force algorithms . 12 3.4 The Onion Skins Algorithm . 13 4 COMPUTING THE LATTICE SIZE WITH RESPECT TO THE CUBE . 15 4.1 Definitions . 15 4.2 Lattice size of polygons with respect to the unit square . 16 4.3 Lattice size in dimension 3 . 19 5 COMPUTING THE LATTICE SIZE WITH RESPECT TO THE SIMPLEX . 29 5.1 Definitions . 29 5.2 Main Results . 30 iii List of Figures Figure 1: Width in the direction a ............................. 6 Figure 2: Newton polytope of f .............................. 7 Figure 3: Lattice size with respect to the standard simplex . 9 Figure 4: Convex hull of interior lattice points . 13 Figure 5: Lattice size with respect to the unit square . 15 Figure 6: Examples of s, u, and t .............................. 31 Figure 7: Example of line segments and simplexes . 32 Figure 8: Region determined by special case . 37 Figure 9: Directions with large width . 38 iv Acknowledgments I would like to thank Jenya Soprunova for her mentorship and support during the research for this project. I would also like to thank Ivan Soprunov for the time he spent teaching and working with me. I am also grateful to the faculty and graduate students in Kent State University’s Math Department for providing a supportive and helpful environment; I had many questions and they were willing to listen and offer their insight. v Chapter 1 Introduction n In this dissertation, we study the lattice size problem. For a lattice polytope P ⊂ R and a n region X ⊂ R of positive Jordan measure, we are interested in determining the smallest integral dilate of X that contains P after an affine unimodular transformation: lsX (P) = minfl 2 N : T(P) ⊂ lX;T 2 AGL(n;Z)g where lX = flx : x 2 Xg and AGL(n;Z) denotes the group of affine unimodular transformations n which act on R (see Chapter 2 and 3 for more details). This problem has been encountered in a number of contexts [1, 2, 14, 16]. There are algorithms [6, 5] that can be used to determine the lattice size, but each of them requires enumerating lattice points in some region. Such an operation is computationally expensive. We will provide new algorithms [11, 12] determine the lattice size for certain choices of X which avoid lattice point enumeration. This dissertation begins with relevant background information on geometry and polytopes (Chapter 2). Chapter 3 gives a detailed history of the lattice size problem and explains some of its connections to algebraic geometry and coding theory. We also give a brief inventory of other algorithms that compute the lattice size. Chapter 4 and 5 constitute the main content of this dis- sertation. These chapters detail new fast algorithms to compute the lattice size with respect to the two- and three-dimensional unit cube and the standard two-dimensional simplex. 1 Chapter 2 Lattices and Polytopes 2.1 Affine and Convex Geometry In this section, it will be convenient to use affine spaces and convex geometry so we collect some definitions here. An affine space is similar to a vector space, but lacks a distinguished origin and has a different algebraic structure. For example, the line L given by y = 1 − x in R2 can be endowed with the structure of an affine space. We note that this set is not closed under addition. However, if we subtract any pair of points a;b 2 L, we get a translation vector. Such a vector v = a − b determines an affine transformation f : L ! L given by f (x) = x + v. The set of all such translations is in fact a vector space. The dimension of an affine space is the dimension of this vector space. Since affine spaces are not closed under all linear combinations, we need a new notion to generate affine spaces. We define an affine combination of a set of points x1;:::;xk to be a linear combination l1x1 + ··· + lkxk such that each li is a real number and l1 + ··· + lk = 1. Suppose n S ⊂ R . The affine hull of S is the set ( k k ) aff(S) = ∑ lixi : k > 0;xi 2 S; ∑ li = 1 . i=1 i=1 This set is smallest affine space which contains S. A convex combination is a linear combination l1x1 +···+lkxk such that li ≥ 0 and l1 +···+ lk = 1. A set that is closed under convex combinations is a convex set. For any convex set X, if x;y 2 X, then the line segment between x and y is also in X. This idea gives an intuitive way to think about convex sets. 2 n Definition The convex hull of a set S ⊂ R is the region ( k k ) conv(S) = ∑ lixi : k > 0;xi 2 S; ∑ li = 1;li ≥ 0 . i=1 i=1 The set S generates its convex hull. This terminology is appropriate since conv(S) is the smallest convex set that contains S [3]. 2.2 Lattices n n A lattice is a discrete subgroup of R that is isomorphic to Z . In this dissertation, we will only n need to work with the lattice Z for an appropriate choice of n. This section focuses on morphisms n n of R that preserve Z . Below we denote the n × n matrices with entries in Z by Mn(Z). n n Definition A unimodular transformation is a linear transformation f : R ! R such that f (v) = Av for some matrix A 2 Mn(Z) with det(A) = ±1. An important property of these transformations is that they map lattice vectors, i.e. vectors in the n n lattice Z , to lattice vectors. In fact, these maps are automorphisms of Z [17]. Among these maps are reflections through the coordinate hyperplanes and certain 90◦ rotations. If A determines a unimodular transformation, then so does A−1. We call the class of matrices which yield such transformations unimodular and denote them by GL(n;Z). There is a larger class of maps which n also preserves Z . n n Definition An affine unimodular transformation f : R ! R is a unimodular transformation com- n posed with a translation by a lattice vector. If A 2 GL(n;Z) and v 2 Z , then the following is such a transformation: f (x) = Ax + v. n We denote the group of affine unimodular transformations on R by AGL(n;Z). 3 2.3 Polytopes A polytope is a particular region of Euclidean space with two equivalent characterizations [17]: 1. A convex hull of finitely many points. 2. A bounded intersection of finitely many half spaces. We will focus on the first characterization. Definition A polytope P is the convex hull of finitely many points fs1;:::;smg. The minimal set V such that conv(V) = P is called the vertex set of P. Elements of V are called vertices. The dimension of a polytope is the dimension of the affine hull of P. This formal definition covers the most commonly encountered polytopes (called polygons in di- mension two) such as squares, triangles, cubes, tetrahedra, etc. n n Definition A lattice polytope is a polytope in R whose vertices are in Z . There are a number of operations on lattice polytopes relevant to this dissertation. Suppose n P ⊂ R is a polytope. We first discuss int(P), the interior of P. If P is full dimensional, dimP = n, n then the interior takes on its typical meaning for a closed set in R . In other situations, it will be more useful to use the relative interior, relint(P): the interior of P in its affine span. n We can translate P by any vector v 2 R : P + v = fx + v : x 2 Pg. We can also scale a polytope by some factor r 2 R and denote this by rP = frx : x 2 Pg. Affine unimodular transformations act on the set of lattice polytopes. In much of what follows, we will work with the equivalence relation induced by this action on lattice polytopes. Let us first n consider very simple lattice polytopes: line segments with endpoints in Z . Such a line segment is called a primitive lattice segment if the only lattice points it contains are its endpoints. We say two segments are unimodularly equivalent if there is an affine unimodular transformation that maps one segment to the other. It is easy to see that any two primitive lattice segments in R2 are 4 unimodularly equivalent. Suppose (a;b) is a primitive lattice segment in R2. Then gcd(a;b) = 1 so there are integers s and t such that as + bt = 1.