Communication Complexity (For Algorithm Designers)
Total Page:16
File Type:pdf, Size:1020Kb
Full text available at: http://dx.doi.org/10.1561/0400000076 Communication Complexity (for Algorithm Designers) Tim Roughgarden Stanford University, USA [email protected] Boston — Delft Full text available at: http://dx.doi.org/10.1561/0400000076 Foundations and Trends R in Theoretical Computer Science Published, sold and distributed by: now Publishers Inc. PO Box 1024 Hanover, MA 02339 United States Tel. +1-781-985-4510 www.nowpublishers.com [email protected] Outside North America: now Publishers Inc. PO Box 179 2600 AD Delft The Netherlands Tel. +31-6-51115274 The preferred citation for this publication is T. Roughgarden. Communication Complexity (for Algorithm Designers). Foundations and Trends R in Theoretical Computer Science, vol. 11, nos. 3-4, pp. 217–404, 2015. R This Foundations and Trends issue was typeset in LATEX using a class file designed by Neal Parikh. Printed on acid-free paper. ISBN: 978-1-68083-115-3 c 2016 T. Roughgarden All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording or otherwise, without prior written permission of the publishers. Photocopying. In the USA: This journal is registered at the Copyright Clearance Cen- ter, Inc., 222 Rosewood Drive, Danvers, MA 01923. Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by now Publishers Inc for users registered with the Copyright Clearance Center (CCC). The ‘services’ for users can be found on the internet at: www.copyright.com For those organizations that have been granted a photocopy license, a separate system of payment has been arranged. Authorization does not extend to other kinds of copy- ing, such as that for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale. In the rest of the world: Permission to pho- tocopy must be obtained from the copyright owner. Please apply to now Publishers Inc., PO Box 1024, Hanover, MA 02339, USA; Tel. +1 781 871 0245; www.nowpublishers.com; [email protected] now Publishers Inc. has an exclusive license to publish this material worldwide. Permission to use this content must be obtained from the copyright license holder. Please apply to now Publishers, PO Box 179, 2600 AD Delft, The Netherlands, www.nowpublishers.com; e-mail: [email protected] Full text available at: http://dx.doi.org/10.1561/0400000076 Foundations and Trends R in Theoretical Computer Science Volume 11, Issues 3-4, 2015 Editorial Board Editor-in-Chief Madhu Sudan Microsoft Research United States Editors Bernard Chazelle Christos Papadimitriou Princeton University University of California, Berkeley Oded Goldreich Prabhakar Raghavan Weizmann Institute Stanford University Shafi Goldwasser Peter Shor MIT & Weizmann Institute MIT Sanjeev Khanna Éva Tardos University of Pennsylvania Cornell University Jon Kleinberg Avi Wigderson Cornell University Princeton University László Lovász Microsoft Research Full text available at: http://dx.doi.org/10.1561/0400000076 Editorial Scope Topics Foundations and Trends R in Theoretical Computer Science publishes surveys and tutorials on the foundations of computer science. The scope of the series is broad. Articles in this series focus on mathematical approaches to topics revolving around the theme of efficiency in com- puting. The list of topics below is meant to illustrate some of the cov- erage, and is not intended to be an exhaustive list. • Algorithmic game theory • Cryptography and information • Computational algebra security • Computational aspects of • Data structures combinatorics and graph theory • Database theory • Computational aspects of • Design and analysis of communication algorithms • Computational biology • Distributed computing • Computational complexity • Information retrieval • Computational geometry • Computational learning • Operations research • Computational Models and • Parallel algorithms Complexity • Quantum computation • Computational Number Theory • Randomness in computation Information for Librarians Foundations and Trends R in Theoretical Computer Science, 2015, Volume 11, 4 issues. ISSN paper version 1551-305X. ISSN online version 1551-3068. Also available as a combined paper and online subscription. Full text available at: http://dx.doi.org/10.1561/0400000076 Foundations and Trends R in Theoretical Computer Science Vol. 11, Nos. 3-4 (2015) 217–404 c 2016 T. Roughgarden DOI: 10.1561/0400000076 Communication Complexity (for Algorithm Designers) Tim Roughgarden Stanford University, USA [email protected] Full text available at: http://dx.doi.org/10.1561/0400000076 Contents Preface2 1 Data Streams: Algorithms and Lower Bounds4 1.1 Preamble........................... 4 1.2 The data stream model ................... 5 1.3 Frequency moments ..................... 7 1.4 Estimating F2: The key ideas ................ 9 1.5 Estimating F0: The high-order bit ............. 16 1.6 Can we do better? ...................... 18 1.7 One-way communication complexity ............ 19 1.8 Connection to streaming algorithms ............ 21 1.9 The disjointness problem .................. 22 1.10 Looking backward and forward ............... 26 2 Lower Bounds for One-Way Communication: Disjointness, Index, and Gap-Hamming 28 2.1 The story so far ....................... 28 2.2 Randomized protocols .................... 30 2.3 Distributional complexity .................. 33 2.4 The Index problem ..................... 35 2.5 Where we’re going ...................... 39 ii Full text available at: http://dx.doi.org/10.1561/0400000076 iii 2.6 The Gap-Hamming problem................ 41 2.7 Lower bound on the one-way communication complexity of Gap-Hamming ....................... 44 3 Lower Bounds for Compressive Sensing 47 3.1 An appetizer: Randomized communication complexity of Equality .......................... 47 3.2 Sparse recovery ....................... 50 3.3 A lower bound for sparse recovery ............. 55 4 Boot Camp on Communication Complexity 68 4.1 Preamble ........................... 68 4.2 Deterministic protocols ................... 69 4.3 Randomized protocols .................... 80 5 Lower Bounds for the Extension Complexity of Polytopes 90 5.1 Linear programs, polytopes, and extended formulations .. 90 5.2 Nondeterministic communication complexity ........ 95 5.3 Extended formulations and nondeterministic communication complexity ................. 99 5.4 A lower bound for the correlation polytope ......... 106 6 Lower Bounds for Data Structures 116 6.1 Preamble ........................... 116 6.2 The approximate nearest neighbor problem ......... 117 6.3 An upper bound: Biased random inner products ...... 118 6.4 Lower bounds via asymmetric communication complexity . 125 7 Lower Bounds in Algorithmic Game Theory 143 7.1 Preamble ........................... 143 7.2 The welfare maximization problem ............. 143 7.3 Multi-party communication complexity ........... 145 7.4 Lower bounds for approximate welfare maximization .... 149 7.5 Lower bounds for equilibria ................. 152 Full text available at: http://dx.doi.org/10.1561/0400000076 iv 8 Lower Bounds in Property Testing 163 8.1 Property testing ....................... 163 8.2 Example: The BLR linearity test .............. 165 8.3 Monotonicity testing: Upper bounds ............ 167 8.4 Monotonicity testing: Lower bounds ............ 175 8.5 A general approach ..................... 179 Acknowledgements 181 References 182 Full text available at: http://dx.doi.org/10.1561/0400000076 Abstract This text collects the lecture notes from the author’s course “Commu- nication Complexity (for Algorithm Designers),” taught at Stanford in the winter quarter of 2015. The two primary goals of the text are: (1) Learn several canonical problems in communication complexity that are useful for proving lower bounds for algorithms (disjoint- ness, index, gap-hamming, etc.). (2) Learn how to reduce lower bounds for fundamental algorithmic problems to communication complexity lower bounds. Along the way, readers will also: (3) Get exposure to lots of cool computational models and some famous results about them — data streams and linear sketches, compressive sensing, space-query time trade-offs in data struc- tures, sublinear-time algorithms, and the extension complexity of linear programs. (4) Scratch the surface of techniques for proving communication com- plexity lower bounds (fooling sets, corruption arguments, etc.). T. Roughgarden. Communication Complexity (for Algorithm Designers). Foundations and Trends R in Theoretical Computer Science, vol. 11, nos. 3-4, pp. 217–404, 2015. DOI: 10.1561/0400000076. Full text available at: http://dx.doi.org/10.1561/0400000076 Preface The best algorithm designers prove both possibility and impossibility results — both upper and lower bounds. For example, every serious computer scientist knows a collection of canonical NP-complete prob- lems and how to reduce them to other problems of interest. Commu- nication complexity offers a clean theory that is extremely useful for proving lower bounds for lots of different fundamental problems. Many of the most significant algorithmic consequences of the theory follow from its most elementary aspects. This document collects the lecture notes from my course “Commu- nication Complexity (for Algorithm Designers),” taught at Stanford in the winter quarter of 2015. The two primary goals of the tutorial are: (1) Learn several