On the Approximability of Injective Tensor Norm Vijay Bhattiprolu
Total Page:16
File Type:pdf, Size:1020Kb
On the Approximability of Injective Tensor Norm Vijay Bhattiprolu CMU-CS-19-115 June 28, 2019 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Venkatesan Guruswami, Chair Anupam Gupta David Woodruff Madhur Tulsiani, TTIC Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Copyright c 2019 Vijay Bhattiprolu This research was sponsored by the National Science Foundation under award CCF-1526092, and by the National Science Foundation under award 1422045. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the U.S. government or any other entity. Keywords: Injective Tensor Norm, Operator Norm, Hypercontractive Norms, Sum of Squares Hierarchy, Convex Programming, Continuous Optimization, Optimization over the Sphere, Approximation Algorithms, Hardness of Approximation Abstract The theory of approximation algorithms has had great success with combinatorial opti- mization, where it is known that for a variety of problems, algorithms based on semidef- inite programming are optimal under the unique games conjecture. In contrast, the ap- proximability of most continuous optimization problems remains unresolved. In this thesis we aim to extend the theory of approximation algorithms to a wide class of continuous optimization problems captured by the injective tensor norm framework. Given an order-d tensor T, and symmetric convex sets C1,... Cd, the injective tensor norm of T is defined as sup hT , x1 ⊗ · · · ⊗ xdi, i x 2Ci Injective tensor norm has manifestations across several branches of computer science, optimization and analysis. To list some examples, it has connections to maximum singu- lar value, max-cut, Grothendieck’s inequality, non-commutative Grothendieck inequality, quantum information theory, k-XOR, refuting random constraint satisfaction problems, tensor PCA, densest-k-subgraph, and small set expansion. So a general theory of its ap- proximability promises to be of broad scope and applicability. We study various important special cases of the problem (through the lens of convex optimization and the sum of squares (SoS) hierarchy) and obtain the following results: - We obtain the first NP-hardness of approximation results for hypercontractive norms. Specifically, we prove inapproximability results for computing the p ! q operator norm (which is a special case of injective norm involving two convex sets) when p ≤ q and 2 62 [p, q]. Towards the goal of obtaining strong inapproximability results for 2 ! q norm when q > 2, we give random label cover (for which polynomial level SoS gaps are available) based hardness results for mixed norms, i.e., 2 ! `q(`q0 ) for some 2 < q, q0 < ¥. - We obtain improved approximation algorithms for computing the p ! q operator norm when p ≥ 2 ≥ q. - We introduce the technique of weak decoupling inequalities and use it to analyze the integrality gap of the SoS hierarchy for the maxima of various classes of polyno- mials over the sphere, namely arbitrary polynomials, polynomials with non-negative coefficients and sparse polynomials. We believe this technique is broadly applicable and could find use beyond optimization over the sphere. We also study how well higher levels of SoS approximate the maximum of a random polynomial over the sphere ([RRS16] concurrently obtained a similar result). Acknowledgements First and foremost I would like to thank my parents and sister for their undying support and patience. I owe a great deal to both of my grandfathers for encouraging me on my path and teaching me in my early years. I am most fortunate to have been advised by Venkat Guruswami. As all of his students would confirm, he adapts to the needs of his students and I can’t think of anything I wish he did differ- ently. He has been an excellent collaborator: he had troves of questions relevant to my background and interests and would constantly make prescient comments about my research directions. Any confidence I have in being able to survive outside the nest, I owe to Venkat’s style of advisory. I spent two wonderful summers (and more) at TTI chicago hosted by Madhur Tulsiani. Mad- hur has always been incredibly generous with his time and knowledge, and has been virtually a second advisor to me. Thanks Madhur! I greatly enjoyed my collaboration and friendship with Euiwoong and Mrinal. I’ve learned so much from both of them and I look forward to our collaborations for years to come. I am very grateful to Assaf Naor, Prasad Raghavendra and Pravesh Kothari for their insightful and influential comments on my research directions. Special thanks to Prasad for hosting me at Berkeley for a few months. I’d like to thank my committee (Venkat, Madhur, David Woodruff and Anupam Gupta) for taking the time and giving me valuable comments. My gratitude to Anupam, Danny and David for helping with Ph.D. program requirements. Ryan and Anupam have given several talks that I found inspiring and they are both amongst my go-to references on pedagogical issues. Special thanks to Anupam for going out of his way many times to tell me how I can improve. Thanks are due to Boris, Tom, Po and Ravi for teaching fun courses. I am grateful to Gary for sparking my interest in spectral theory. My five years at CMU have been the most enjoyable in my lifetime so far. A big part of this is due to the wonderful group of peers I had. While it would be impossible to list them all, some memories that stand out are: ploughing through coursework with David Wajc and Arash; working at coffee shops with Nic, Naama and Nika; working nights at GHC with Yu, Daniel and Lei; ping pong with David Wajc, John, Ainesh, Anish, Goran, Angela, Pedro, Sahil, Sai and David Kurokawa; climbing with Colin, Roie, Aria, Greg, Anson, Andrei, Ellis, and Woneui; classical music concerts with John, Yu and Ellen; board games with Sol, Ameya and many others; fun conversations with Amir, Ilqar, Laxman, Guru, Jakub, Alex, Sidhanth and David Witmer. Last but not least I would like to thank Deb Cavlovich without whom I imagine SCS would cease functioning. i Contents I Preamble1 1 Introduction2 1.1 Convex Programming Relaxations........................5 1.2 Sum of Squares Hierarchy.............................5 1.3 Brief Summary of Contributions.........................6 1.4 Chapter Credits...................................7 1.5 Organization.....................................7 2 Detailed Results8 2.1 `p ! `q Operator Norms..............................8 2.1.1 Hypercontractive norms, Small-Set Expansion, and Hardness.....9 2.1.2 The non-hypercontractive case....................... 11 2.2 Polynomial Optimization over the Sphere.................... 15 II Degree-2 (Operator Norms) 18 3 Preliminaries (Normed Spaces) 19 3.1 Vectors........................................ 19 3.2 Norms........................................ 19 3.3 `p Norms....................................... 20 3.4 Operator Norm................................... 20 3.5 Type and Cotype.................................. 21 3.6 p-convexity and q-concavity............................ 23 3.7 Convex Relaxation for Operator Norm...................... 23 3.8 Factorization of Linear Operators......................... 25 4 Hardness results for p!q norm 27 4.1 Proof Overview................................... 28 4.2 Preliminaries and Notation............................ 29 ii 4.2.1 Fourier Analysis.............................. 30 4.2.2 Smooth Label Cover............................ 31 4.3 Hardness of 2!r norm with r < 2........................ 31 4.3.1 Reduction and Completeness....................... 32 4.3.2 Soundness.................................. 33 4.4 Hardness of p!q norm............................... 35 4.4.1 Hardness for p ≥ 2 ≥ q .......................... 36 4.4.2 Reduction from p!2 norm via Approximate Isometries....... 36 4.4.3 Derandomized Reduction......................... 38 4.4.4 Hypercontractive Norms Productivize.................. 40 4.4.5 A Simple Proof of Hardness for the Case 22 / [q, p] ........... 41 4.5 Dictatorship Test.................................. 45 5 Algorithmic results for p!q norm 48 5.1 Proof overview................................... 48 5.2 Relation to Factorization Theory......................... 50 5.3 Approximability and Factorizability....................... 51 5.4 Notation....................................... 51 5.5 Analyzing the Approximation Ratio via Rounding............... 52 5.5.1 Krivine’s Rounding Procedure...................... 52 5.5.2 Generalizing Random Hyperplane Rounding – Hölder Dual Round- ing...................................... 53 5.5.3 Generalized Krivine Transformation and the Full Rounding Proce- dure..................................... 54 5.5.4 Auxiliary Functions............................ 55 5.5.5 Bound on Approximation Factor..................... 55 5.6 Hypergeometric Representation.......................... 57 5.6.1 Hermite Preliminaries........................... 57 5.6.2 Hermite Coefficients via Parabolic Cylinder Functions........ 59 5.6.3 Taylor Coefficients of f........................... 62 5.7 Bound on Defect................................... 64 5.7.1 Behavior of The Coefficients of The Inverse Function......... 64 5.7.2 Bounding Inverse Coefficients...................... 66 5.8 Factorization of Linear Operators......................... 75 5.8.1 Integrality Gap Implies Factorization Upper Bound.......... 75 5.8.2