Fourier Analysis on the Hypercube, the Coefficient Problem, And
Total Page:16
File Type:pdf, Size:1020Kb
Fourier Analysis on the Hypercube, the Coefficient Problem, and Applications by Ganesh Ajjanagadde S.B., Massachusetts Institute of Technology (2015) M.Eng., Massachusetts Institute of Technology (2016) Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY May 2020 ○c Massachusetts Institute of Technology 2020. All rights reserved. Author................................................................ Department of Electrical Engineering and Computer Science April 28, 2020 Certified by. Gregory Wornell Professor of Electrical Engineering and Computer Science Thesis Supervisor Certified by. Henry Cohn Senior Principal Researcher, Microsoft Research New England Thesis Supervisor Accepted by . Leslie A. Kolodziejski Professor of Electrical Engineering and Computer Science Chair, Department Committee on Graduate Students 2 Fourier Analysis on the Hypercube, the Coefficient Problem, and Applications by Ganesh Ajjanagadde Submitted to the Department of Electrical Engineering and Computer Science on April 28, 2020, in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Computer Science Abstract In this dissertation, we primarily study some problems that revolve around Fourier analysis. More specifically we focus on the magnitudes of the frequency components. Firstly, we perform a study on the hypercube. It is well known that the Delsarte lin- ear programming bounds provide rich information on the magnitudes of the Fourier coefficients, grouped by Hamming weight. Classically, such information is primarily used to attack coding problems, where the objective is to maximize cardinality of a subset of a metric space subject to a minimum distance constraint. Here, we use it to study anticoding problems, where the objective is to maximize cardinality of a subset of a metric space subject to a maximum distance (diameter) constraint. One moti- vation for such study is the problem of finding memories that are cheap to update, where the cost of an update is a function of the distance in the metric space. Such a view naturally supports the study of different cost functions going beyond hard diam- eter constraints. We work accordingly with different cost functions, with a particular emphasis on completely monotonic functions. Our emphasis is on the phenomenon of “universal optimality”, where the same subset (anticode) simultaneously optimizes a wide range of natural cost functions. Among other things, our work here gives some answers to a question in computer science, namely finding Boolean functions with maximal noise stability subjected to an expected value constraint. Secondly, we work with Fourier analysis on the integers modulo a number by draw- ing upon Nazarov’s general solution to the “coefficient problem”. Roughly speaking, the coefficient problem asks one to construct time domain signals with prescribed magnitudes of frequency components, subject to certain natural constraints on the signal. In particular, Nazarov’s solution works with lp constraints in time. This solution to the coefficient problem allows us to give an essentially complete resolu- tion to the mathematical problem of designing optimal coded apertures that arises in computational imaging. However, the resolution we provide is for an l constraint on the aperture, corresponding to partial occlusion. We believe it is important1 to also examine a binary valued (f0; 1g) constraint on the aperture as one does not need to synthesize partial occluders for such apertures. We therefore provide some 3 preliminary results as well as directions for future research. Finally, inspired by the recent breakthroughs in understanding the d = 8; 24 cases of sphere packing and universal optimality in Rd, we attempt to show that the associated lattices (E8 and the Leech lattice for d = 8; 24 respectively) are also optimal for the problem of vector quantization in the sense of minimizing mean squared error. Accordingly, we develop a dispersion and anticoding based approach to lower bounds on the mean squared error. We also generalize Tóth’s method, which shows optimality of the hexagonal lattice quantizer for d = 2, to arbitrary d. To the best of our knowledge, these methods give the first rigorous improved lower bounds for the mean squared error for all large enough d since the work of Zador over 50 years ago. Thesis Supervisor: Gregory Wornell Title: Professor of Electrical Engineering and Computer Science Thesis Supervisor: Henry Cohn Title: Senior Principal Researcher, Microsoft Research New England 4 Acknowledgments It is a Herculean task to express my gratitude to all who have contributed directly and indirectly to my adventures in graduate school and in this dissertation. I can only offer a few succinct remarks here. For those of you not explicitly mentioned here:first off, I am confident that you are already aware of my deep respect for you, andthatyou would not derive much meaning or pleasure from an explicit acknowledgement here. Secondly, I am grateful for the very fact that you are examining this dissertation. I hope that you find it at best illuminating, and at worst somewhat entertaining. Keeping this in my mind, I now turn to some of the most important people, hoping that you understand how much you have meant to me. First, I thank my parents, Venkataramana and Vijaya Ajjanagadde. To put it simply: I am who I am because of you; any merits I might have are due to you, and any demerits are mine alone. Second, I thank my thesis advisers, Profs. Gregory Wornell and Henry Cohn. Both of you have not only been extremely patient, supportive, infectiously optimistic, and insightful with respect to research, but have also extended it to my development as a person. It was an honor to be your student. Third, I thank Prof. Yury Polyanskiy, my thesis committee member. It is pretty safe to say that I was forged as a researcher in his smithy as an undergraduate. I am also grateful to him for exposing me to the wonders of the Russian intelligentsia. Fourth, I acknowledge several professors with whom I have had stimulating inter- actions over the past eight years here at MIT: Profs. Guy Bresler, Polina Golland, Alexandre Megretski, Elchanan Mossel, John Tsitsiklis, George Verghese, Alan Will- sky, and Lizhong Zheng. Fifth, I acknowledge the friends who have served as a bedrock during my time here at MIT: Mohamed AlHajri, Nirav Bhan, Kishor Bhat, Austin Collins, Matthew de Courcy-Ireland, Igor Kadota, Pranav Kaundinya, Joshua Ka-Wing Lee, Eren Kizildag, Suhas Kowshik, Fabián Kozynski, Pavitra Krishnaswamy, Ashwin Kumar, Tarek Lahlou, Bhavesh Lalwani, Anuran Makur, Dheeraj Nagaraj, Deepak Narayanan, 5 James Noraky, Or Ordentlich, Rishi Patel, David Qiu, Govind Ramnarayan, Ankit Rawat, Arman Rezaee, Hajir Roozbehani, Tuhin Sarkar, Anjan Soumyanarayanan, Harihar Subramanyam, James Thomas, Christos Thrampoulidis, Aditya Venkatra- mani, and Adam Yedidia. Sixth, I acknowledge the outstanding labmates (past and present) who have made the Signals, Information, and Algorithms Lab a special place: Toros Arikan, Yuheng Bu, Qing He, Tejas Jayashankar, Gauri Joshi, Gary Lee, Emin Martinian, Safa Medin, Gal Shulkind, Atulya Yellepeddi, and Xuhong Zhang. I also acknowledge the always friendly and helpful administrative assistant Tricia O’Donnell. Finally, I thank my extended family. They have been a very deep foundation during the times when I could not see the light at the end of the tunnel. In particular, I dedicate this thesis to my grandmothers Parvathi Ajjanagadde and Shankari Bhat, who instilled a deep love for education and knowledge in their descendants in spite of very difficult circumstances. The material in this dissertation is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. CCF- 1816209. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation. Much of the work presented in Chapter 4 was performed while I was at Microsoft Research New England, whose support I gratefully acknowledge. 6 Contents 1 Introduction 13 1.1 Motivation . 13 1.1.1 Hamming space . 14 1.1.2 Discrete Fourier transforms . 15 1.1.3 Euclidean space . 16 1.1.4 General remarks . 17 2 Linear Programming Bounds and Anticodes in Hamming Space 19 2.1 Introduction . 19 2.2 Main results . 24 2.2.1 Linear programming and isodiametry in Hamming space . 24 2.2.2 Universal optimality for special subcubes . 24 2.2.3 A mean value theorem for noise stability . 27 2.2.4 Large n and balls versus subcubes . 28 2.3 Linear programming and isodiametry in Hamming space . 28 2.3.1 LP bounds and Fourier analysis on Hamming space . 28 2.3.2 Proof of Theorem 1 . 39 2.4 Universal optimality of some subcubes . 47 2.5 Connection to maximum noise stability . 56 2.6 A mean value theorem for noise stability . 60 2.7 Large n and balls versus subcubes . 62 2.8 Open problems . 76 7 3 Near-Optimal Coded Apertures for Imaging via Nazarov’s Theorem 79 3.1 Introduction . 80 3.2 Model . 87 3.2.1 Background on linear estimation . 88 3.2.2 Model clarifications and intuition . 91 3.3 Results . 92 3.3.1 Lower bound . 92 3.3.2 Upper bound . 96 3.3.3 Greedy algorithm . 110 3.3.4 Simulations . 110 3.4 Discussion and Future Work . 111 4 New lower bounds for the mean squared error of vector quantizers119 4.1 Introduction . 119 4.2 Main results . 123 4.3 Proofs . 125 4.3.1 Rederivation of Zador’s lower bound . 126 4.3.2 The general case . 127 4.3.3 Tóth/Newman’s hexagon theorem . 132 4.3.4 Conjectured bound for lattices . 136 4.3.5 Towards a proof for lattices . 140 4.4 Discussion and Future Work . 144 5 Conclusion and Future Directions 147 Appendices 150 A Proofs for Chapter 3 151 8 List of Figures 3-1 A pinhole camera.