Sparse Polynomial Interpolation and Testing
Total Page:16
File Type:pdf, Size:1020Kb
Sparse Polynomial Interpolation and Testing by Andrew Arnold A thesis presented to the University of Waterloo in fulfillment of the thesis requirement for the degree of Doctor of Philsophy in Computer Science Waterloo, Ontario, Canada, 2016 © Andrew Arnold 2016 I hereby declare that I am the sole author of this thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. ii Abstract Interpolation is the process of learning an unknown polynomial f from some set of its evalua- tions. We consider the interpolation of a sparse polynomial, i.e., where f is comprised of a small, bounded number of terms. Sparse interpolation dates back to work in the late 18th century by the French mathematician Gaspard de Prony, and was revitalized in the 1980s due to advancements by Ben-Or and Tiwari, Blahut, and Zippel, amongst others. Sparse interpolation has applications to learning theory, signal processing, error-correcting codes, and symbolic computation. Closely related to sparse interpolation are two decision problems. Sparse polynomial identity testing is the problem of testing whether a sparse polynomial f is zero from its evaluations. Sparsity testing is the problem of testing whether f is in fact sparse. We present effective probabilistic algebraic algorithms for the interpolation and testing of sparse polynomials. These algorithms assume black-box evaluation access, whereby the algorithm may specify the evaluation points. We measure algorithmic costs with respect to the number and types of queries to a black-box oracle. Building on previous work by Garg–Schost and Giesbrecht–Roche, we present two methods for the interpolation of a sparse polynomial modelled by a straight-line program (SLP): a sequence of arithmetic instructions. We present probabilistic algorithms for the sparse interpolation of an SLP, with cost softly-linear in the sparsity of the interpolant: its number of nonzero terms. As an application of these techniques, we give a multiplication algorithm for sparse polynomials, with cost that is sensitive to the size of the output. Multivariate interpolation reduces to univariate interpolation by way of Kronecker substitu- tion, which maps an n-variate polynomial f to a univariate image with degree exponential in n. We present an alternative method of randomized Kronecker substitutions, whereby one can more efficiently reconstruct a sparse interpolant f from multiple univariate images of considerably reduced degree. In error-correcting interpolation, we suppose that some bounded number of evaluations may be erroneous. We present an algorithm for error-correcting interpolation of polynomials that are sparse under the Chebyshev basis. In addition we give a method which reduces sparse Chebyshev- basis interpolation to monomial-basis interpolation. Lastly, we study the class of Boolean functions that admit a sparse Fourier representation. We give an analysis of Levin’s Sparse Fourier Transform algorithm for such functions. Moreover, we give a new algorithm for testing whether a Boolean function is Fourier-sparse. This method reduces sparsity testing to homomorphism testing, which in turn may be solved by the Blum– Luby–Rubinfeld linearity test. iii Acknowledgements I would like to acknowledge some of the people who supported me throughout my studies: • My supervisor, Mark Giesbrecht. Mark has been very generous with his time and resources; introduced inspiring problems to me; and gave me the freedom and support to pursue my research. Mark has provided me direction and opened many doors for me along my career path. Thank you, Mark. • My friend Dan Roche, for our many collaborations. It is always a pleasure discussing mathe- matics with Dan. I hope we continue to work together on interesting problems. • Eric Blais and Erich Kaltofen, for the opportunity to work with each of them on rewarding projects that became chapters of my thesis. • My committee, for their thoughtful questions, and their careful reading of my thesis draft. In addition to other committee members mentioned herein I would like to name my internal- external examiner Kevin Hare; my external examiner Jeremy Johnson; and Eric´ Schost, whose work on interpolation was a starting point for my doctoral research. Thanks as well to Evelyne Hubert and Cl´ement Pernet for sharing stimulating questions and ideas. • My Master’s supervisor, Michael Monagan, without whose direction I would not have pursued doctoral studies at Waterloo; and Peter Borwein, who first encouraged me to pursue research. • To my friends at the Symbolic Computation Group (SCG) lab, who made it a fun and stimu- lating place to work at: Curtis Bright, Reinhold Burger, Shaoshi Chen, Mustafa Elshiek, Joseph Haraldson, Albert Heinle, Ilias Kosteras, Winnie Lam, Roman Lebreton, Steve Melczer, Vijay Menon, Andy Novocin, Colton Pauderis, and Nam Phan. I also thank SCG faculty George Labahn and Arne Storjohann for the times they’ve given me valued professional advice. • For their generous financial support: the National Sciences and Engineering Research Council of Canada (NSERC), the Ontario Graduate Scholarship (OGS) program, the David R. Cheriton School of Computer Science, the University of Waterloo, and the Fields Institute. • My friends, who enriched my time spent in Waterloo and Toronto: Parsiad Azimzadeh, Cecylia Bocovich, Duane Bobbsemple, Hicham El-Zein, Sam Gentile, Hella Hoffmann, Mark Horton, Carol Jung, Stephen Kiazyk, Erik Postma, Shadab Rashid, Johl Ringuette, Dean Shaft, Valerie Sugarman, Jack Thomas, Oliver Trujillo, The Clockwork (my ultimate team), and the David R. Cheriton School of Hockey Superfriends (my intramural hockey team); also my out-of- town friends Kyle Allesia, Jamieson Anderson, Colin Beaumont, Nate Hapke, Reuben Heredia, Athena Li, Chelsea Richards, Mike Rozen, and Rob Szolomicki, for being a phone call away whenever I needed a laugh. Lastly, I want to thank my family: my parents, Dave and Marie; my brother Thomas; and my sister-in-law Yoshiko. I could not have written this thesis without their encouragement and sup- port. iv To my parents and my big brother Thomas. v Table of Contents Author’s Declaration ii Abstract iii Acknowledgements iv Dedication v List of Tables xi List of Figures xii List of Algorithms xiii 1 Introduction 1 1.1 Preliminaries ..................................... 2 1.2 Model of Computation ................................ 3 1.3 Sparse and dense polynomial representations ................... 6 1.3.1 Vector polynomials .............................. 8 1.4 Models of polynomial evaluation .......................... 9 1.4.1 Black-box polynomials ............................ 9 1.4.2 Extended-black-box polynomials ...................... 10 1.4.3 Straight-line programs ............................ 12 1.5 Organization of thesis ................................ 14 vi 2 Algorithmic tools 16 2.1 Integer, dense, modular ring, and finite field arithmetic .............. 17 2.1.1 Integer and modular arithmetic ....................... 17 2.1.2 Dense polynomial arithmetic ........................ 18 2.1.3 Finite field operations ............................ 19 2.1.4 Chinese remaindering and simultaneous modular reduction ....... 20 2.1.5 Polynomial factorization via Hensel lifting ................. 20 2.1.6 Linear algebra ................................ 21 2.2 Data structures .................................... 23 2.2.1 Constructing dictionaries of terms ..................... 24 2.3 Probabilistic inequalities ............................... 26 2.3.1 Amplifying probabilistic algorithms ..................... 27 2.4 Selecting primes ................................... 28 2.4.1 Constructing the first k primes via sieve methods ............. 28 2.4.2 Selecting random primes from a specified interval ............. 29 2.4.3 Constructing elements in Zq of specified order ............... 33 2.4.4 Selecting primes in arithmetic progressions ................ 33 2.5 Probabilistic black-box polynomial identity testing ................. 37 2.6 Kronecker Substitution ................................ 40 2.7 Prony’s algorithm for interpolating black-box polynomials ............ 42 2.7.1 Linearly generated sequences ........................ 42 2.7.2 Determining the minimal generator ..................... 45 2.7.3 Prony’s algorithm ............................... 46 2.7.4 Decoding BCH codes and Prony’s algorithm over finite fields ....... 47 2.7.5 The Ben-Or–Tiwari Algorithm ........................ 48 2.8 Early termination, sparsity testing, and sparsity estimation ............ 49 vii 3 Sparse interpolation of straight-line programs 52 3.1 Introduction ...................................... 52 3.2 Preliminaries ..................................... 53 3.2.1 Extended black box queries ......................... 53 3.2.2 Hashing exponents .............................. 55 3.2.3 Organization of chapter and summary of results .............. 56 3.3 Previous Work ..................................... 57 3.3.1 The Garg–Schost algorithm ......................... 57 3.3.2 Probabilistic Garg–Schost .......................... 60 3.3.3 Related work: sparse polynomial identity testing of SLPs ......... 61 3.3.4 The Giesbrecht–Roche “diversified” method ................ 63 3.4 Iterative sparse interpolation ............................ 65 3.4.1 Allowing for some collisions with σ-support primes ............ 66 3.4.2 Deceptive terms ................................ 68 3.4.3 Interpolation