DOCSLIB.ORG

On the Concrete Hardness of Learning with Errors

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
On the Concrete Hardness of Learning with Errors on the concrete hardness of learning with errors Martin R. Albrecht @martinralbrecht joint work with Rachel Player and Sam Scott ACE Seminar, UCL. May 7, 2015 Information Security Group Royal Holloway, University of London 1 outline Learning With Errors Strategies and Algorithms Lattice Reduction Estimator Conclusion 2 outline Learning With Errors Strategies and Algorithms Lattice Reduction Estimator Conclusion 3 lattice-based cryptography Lattice A lattice is a discrete additive subgroup of Rm. Basis f g Rm Let B = b1; :::;Pbm be a set of m linearly independent vectors in . f m j 2 Zg Then L(B) = i=1 xibi xi is the lattice generated by this basis. Dual Lattice Given a lattice L(B) ⊂ Rm, define its dual as fx 2 Rm j xB 2 Zmg. f 2 Zm j ≡ g We’ll only use scaled-by-q dual lattices, i.e. x q xB 0 4 learning with errors The Learning with Errors (LWE) problem was defined by Oded Regev [Reg05]. ∙ Suppose a public matrix A and a secret vector s. ∙ If we were also given b = As we could compute s by linear algebra. ∙ Now imagine this is noisy: c = As + e with e small. ∙ From A and c can we find s? Was c even computed this way? 5 learning with errors 2 Zm 2 Zm×n 2 Zn 2 Zm×` Given (A; c) with c q , A q , s q and e q do we have 0 1 0 1 0 1 n ! B C B C B C B C B C B C B C B C 0 1 B C B C B C B C B C B C B C B C B C B C B C B c C = B A C × @ s A + B e C B C B C B C B C B C B C B C B C B C @ A @ A @ A U Zm or c $ ( q ). 6 parameters ∙ Parameters are: 1 ∙ dimension n, ≈ 2 0:8 ∙ modulus q (e.g. q n ),p ∙ noise size α (e.g. αq ≈ n), ) : x 0 6 ( ∙ number of samples m. Pr ≈ 0:4 ∙ Elements of A; s; e; c are in Zq. 0:2 ∙ e is sampled from a discrete 0 Gaussian with width −10 −5 0 5 10 αq σ = p : x 2π 7 search and decision Search LWE From samples (A; c) recover s. Decision LWE Determine if samples (A; c) are LWE or uniformly random. These problems are polynomial-time equivalent. 8 lwe normal form Given samples h i 2 Zn × Z (a; c) = (a; a; s + e) q q U Zn 2 Zn with a ( q), e Dαq;0 and s q, we can construct samples h i 2 Zn × Z (a; c) = (a; a; e + e) q q U Zn with a ( q), e Dαq;0 and e such that all components ei Dαq;0 in polynomial time. 9 switching moduli h i 2 Zn × Z Let (a; c) = (a; a; s + e) q q be an LWE sample and r 2π n σ p ≈ · s ; 12 α where σs is the standard deviation of components of the secret s. If p < q then (⌊ ⌉ ⌊ ⌉) p p · a ; · c in Zn × Z q q p p followsp a distribution close to an LWE distribution with parameters n; 2α; p. 10 why care? Learning With Errors ∙ is assumed to be a hard problem like discrete logarithms, factoring, etc. ∙ reduces to hard problems on lattices, such as GapSVP. ∙ is assumed to have resistance against quantum computers, unlike discrete logarithms and factoring. ∙ is remarkably versatile for constructing cryptographic schemes. 11 applications Identity-based encryption [GPV08] Ciphertexts are of the form (p; c) = (As + e; u · s + e + b · bq=2c) where H(id) = u = xTA is the public key for the private key x. Decryption is done by c − hx; pi = −⟨x; ei + e + b · bq=2c: 12 applications Fully homomorphic encryption [BV11, AFFP11] Think of LWE encryptions h i · b c (ai; ci) = (ai; ai; s + ei + bi q=2 ) as noisy linear polynomials X − ci + aijxj: Add, multiply and decrypt as usual. 13 how hard is lwe? Given n (and α, q) how many operations does it take to solve? ∙ Problem 1. Algorithms/attacks are not well understood in terms of concrete running times. ∙ Runtimes are given asymptotically. ∙ Algorithms are better in practice than the theoretical bounds. ∙ Many heuristic assumptions. ∙ Problem 2. Many variables ∙ dimension, modulus, secret size ∙ distribution of the secret ∙ number of samples available to an attacker ∙ variants of the problem (e.g. small secrets, BinaryError-LWE) 14 what do people do currently? Often, in the literature, the following assumptions were made when estimating concrete security of an LWE-based scheme: ∙ the best attack is a lattice-based distinguishing attack; ∙ BKZ runs in roughly the time given in [LP11]; ∙ the use of a small secret makes no difference for attacks. All three of these assumptions turn out not to be correct. 15 so, what did we do? ∙ Overview the strategies for attacking LWE. ∙ Analyse and present running times. ∙ Produce concrete estimates for attack timings for parameters sets. The estimation code is available as a Sage module. 16 outline Learning With Errors Strategies and Algorithms Lattice Reduction Estimator Conclusion 17 strategies for solving lwe BDD in L(A) SIS in dual of L(A) Recover s Dec Kannan Distinguish BKW Guess Arora-GB Lattice Reduction SVP Oracle 18 arora-gb ∙ The error is from a small 1 subset of Zq, say, (−τ · σ; : : : ; τ · σ) 0:8 ∙ Each candidate gives rise 0:6 to one linear equation. ∙ Construct equations of 0:4 degree 2 τ · σ + 1 0:2 encoding that one of these linear equations 0 must hold. −15 −10 −5 0 5 10 15 ∙ Solve the system using Gröbner bases. 19 arora-gb p Arora-Ge (Linearisation) with σ = n ( ) O 2! n log(8n log n)−!n log n p Gröbner Bases with σ = n ( ) O 22:82 ! n under some regularity assumption. M.A., Carlos Cid, Jean-Charles Faugère, and Ludovic Perret. Algebraic algorithms for LWE. Cryptology ePrint Archive, Report 2014/1018, 2014. http://eprint.iacr.org/2014/1018. 20 what is sis? Short Integer Solutions (SIS) Given q 2 Z, a matrix B, and t < q; find y with 0 < kyk ≤ t and yB ≡ 0 (mod q): ∙ Recall the dual lattice: L∗ = fx 2 Rm j xB 2 Zmg. ∙ Then the scaled dual lattice, qL∗ has the property that xB ≡ 0 (mod q) for all x 2 qL∗. ∙ Therefore, a short vector of qL∗ is equivalent to solving SIS on B. 21 strategy ∙ Find a short y solving SIS on A. ∙ Given LWE samples A; c where either c = As + e or c uniformly random. ∙ Compute hy; ci. ∙ If c = As + e, then hy; ci = hyA; si + hy; ei ≡ hy; ei (mod q). ∙ If c is uniformly random, so is hy; ci. ∙ If y is sufficiently short, since e is also small, then hy; ei will also be short, and can be distinguished from uniform values. 22 distinguish (lattice reduction) A reduced lattice basis is made of short vectors, in particular the first vector. 1. Construct a basis of the dual from the instance. 2. Feed to a lattice reduction algorithm to obtain short vectors vi. 3. Check if vi A are small. Daniele Micciancio and Oded Regev. Lattice-based cryptography. In Bernstein et al. [BBD09], pages 147–191. 23 bkw algorithm We revisit Gaussian elimination: 0 1 a a a ··· a c B 11 12 13 1n 1 C B ··· C B a21 a22 a23 a2n c2 C B . C @ . .. A am1 am2 am3 ··· amn cm 0 1 a a a ··· a ha ; si + e B 11 12 13 1n 1 1 C B ··· h ; i + C ? B a21 a22 a23 a2n a2 s e2 C = B . C @ . .. A am1 am2 am3 ··· amn ham; si + em 24 bkw algorithm 0 1 a a a ··· a ha ; si + e B 11 12 13 1n 1 1 C a21 B 0 a~22 a~23 ··· a~2n ha~2; si + e2 − e1 C ) B a11 C B . C @ . .. A 0 a~ a~ ··· a~ ha~ ; si + e − am1 e m2 m3 mn m m a11 1 ∙ ai1 is essentially random in Z wiping all “smallness”. a11 q ∙ If ai1 is 1 noise-size doubles because of the addition. a11 25 bkw algorithm We considering a ≈ log n ‘blocks’ of b elements each. 0 1 a a a ··· a c B 11 12 13 1n 0 C B ··· C B a21 a22 a23 a2n c1 C B . C @ . .. A am1 am2 am3 ··· amn cm 26 bkw algorithm For each block we build a table of all qb possible values indexed by Zb q. 2 3 −⌊ q c −⌊ q c ··· 2 2 t13 t1n ct;0 6 q q 7 6 −⌊ c −⌊ c + 1 t23 ··· t2n ct;1 7 0 6 2 2 7 T = 6 . 7 4 . .. 5 b q c b q c ··· 2 2 tq23 tq2n ct;q2 2 Zb For each z q find row in A which contains z as a subvector at the target indices. 27 bkw algorithm 0 1 a a a ··· a c B 11 12 13 1n 0 C B ··· C B a21 a22 a23 a2n c1 C B . C @ . .. A a a a ··· a c 2 m1 m2 m3 mn m 3 −⌊ q c −⌊ q c t ··· t c 6 2 2 13 1n t;0 7 6 −⌊ q c −⌊ q c + ··· 7 6 2 2 1 t23 t2n ct;1 7 + 6 .
Load more

Recommended publications
  • The Next Digital Decade Essays on the Future of the Internet
    THE NEXT DIGITAL DECADE ESSAYS ON THE FUTURE OF THE INTERNET Edited by Berin Szoka & Adam Marcus THE NEXT DIGITAL DECADE ESSAYS ON THE FUTURE OF THE INTERNET Edited by Berin Szoka & Adam Marcus NextDigitalDecade.com TechFreedom techfreedom.org Washington, D.C. This work was published by TechFreedom (TechFreedom.org), a non-profit public policy think tank based in Washington, D.C. TechFreedom’s mission is to unleash the progress of technology that improves the human condition and expands individual capacity to choose. We gratefully acknowledge the generous and unconditional support for this project provided by VeriSign, Inc. More information about this book is available at NextDigitalDecade.com ISBN 978-1-4357-6786-7 © 2010 by TechFreedom, Washington, D.C. This work is licensed under the Creative Commons Attribution- NonCommercial-ShareAlike 3.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/3.0/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105, USA. Cover Designed by Jeff Fielding. THE NEXT DIGITAL DECADE: ESSAYS ON THE FUTURE OF THE INTERNET 3 TABLE OF CONTENTS Foreword 7 Berin Szoka 25 Years After .COM: Ten Questions 9 Berin Szoka Contributors 29 Part I: The Big Picture & New Frameworks CHAPTER 1: The Internet’s Impact on Culture & Society: Good or Bad? 49 Why We Must Resist the Temptation of Web 2.0 51 Andrew Keen The Case for Internet Optimism, Part 1: Saving the Net from Its Detractors 57 Adam Thierer CHAPTER 2: Is the Generative
    [Show full text]
  • The Flajolet-Martin Sketch Itself Preserves Differential Privacy: Private Counting with Minimal Space
    The Flajolet-Martin Sketch Itself Preserves Differential Privacy: Private Counting with Minimal Space Adam Smith Shuang Song Abhradeep Thakurta Boston University Google Research, Brain Team Google Research, Brain Team [email protected] [email protected] [email protected] Abstract We revisit the problem of counting the number of distinct elements F0(D) in a data stream D, over a domain [u]. We propose an ("; δ)-differentially private algorithm that approximates F0(D) within a factor of (1 ± γ), and with additive error of p O( ln(1/δ)="), using space O(ln(ln(u)/γ)/γ2). We improve on the prior work at least quadratically and up to exponentially, in terms of both space and additive p error. Our additive error guarantee is optimal up to a factor of O( ln(1/δ)), n ln(u) 1 o and the space bound is optimal up to a factor of O min ln γ ; γ2 . We assume the existence of an ideal uniform random hash function, and ignore the space required to store it. We later relax this requirement by assuming pseudo- random functions and appealing to a computational variant of differential privacy, SIM-CDP. Our algorithm is built on top of the celebrated Flajolet-Martin (FM) sketch. We show that FM-sketch is differentially private as is, as long as there are p ≈ ln(1/δ)=(εγ) distinct elements in the data set. Along the way, we prove a structural result showing that the maximum of k i.i.d. random variables is statisti- cally close (in the sense of "-differential privacy) to the maximum of (k + 1) i.i.d.
    [Show full text]
  • Hardness of Non-Interactive Differential Privacy from One-Way
    Hardness of Non-Interactive Differential Privacy from One-Way Functions Lucas Kowalczyk* Tal Malkin† Jonathan Ullman‡ Daniel Wichs§ May 30, 2018 Abstract A central challenge in differential privacy is to design computationally efficient non-interactive algorithms that can answer large numbers of statistical queries on a sensitive dataset. That is, we would like to design a differentially private algorithm that takes a dataset D Xn consisting of 2 some small number of elements n from some large data universe X, and efficiently outputs a summary that allows a user to efficiently obtain an answer to any query in some large family Q. Ignoring computational constraints, this problem can be solved even when X and Q are exponentially large and n is just a small polynomial; however, all algorithms with remotely similar guarantees run in exponential time. There have been several results showing that, under the strong assumption of indistinguishability obfuscation (iO), no efficient differentially private algorithm exists when X and Q can be exponentially large. However, there are no strong separations between information-theoretic and computationally efficient differentially private algorithms under any standard complexity assumption. In this work we show that, if one-way functions exist, there is no general purpose differen- tially private algorithm that works when X and Q are exponentially large, and n is an arbitrary polynomial. In fact, we show that this result holds even if X is just subexponentially large (assuming only polynomially-hard one-way functions). This result solves an open problem posed by Vadhan in his recent survey [Vad16]. *Columbia University Department of Computer Science.
    [Show full text]
  • Communication Complexity (For Algorithm Designers)
    Full text available at: http://dx.doi.org/10.1561/0400000076 Communication Complexity (for Algorithm Designers) Tim Roughgarden Stanford University, USA [email protected] Boston — Delft Full text available at: http://dx.doi.org/10.1561/0400000076 Foundations and Trends R in Theoretical Computer Science Published, sold and distributed by: now Publishers Inc. PO Box 1024 Hanover, MA 02339 United States Tel. +1-781-985-4510 www.nowpublishers.com [email protected] Outside North America: now Publishers Inc. PO Box 179 2600 AD Delft The Netherlands Tel. +31-6-51115274 The preferred citation for this publication is T. Roughgarden. Communication Complexity (for Algorithm Designers). Foundations and Trends R in Theoretical Computer Science, vol. 11, nos. 3-4, pp. 217–404, 2015. R This Foundations and Trends issue was typeset in LATEX using a class file designed by Neal Parikh. Printed on acid-free paper. ISBN: 978-1-68083-115-3 c 2016 T. Roughgarden All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, mechanical, photocopying, recording or otherwise, without prior written permission of the publishers. Photocopying. In the USA: This journal is registered at the Copyright Clearance Cen- ter, Inc., 222 Rosewood Drive, Danvers, MA 01923. Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by now Publishers Inc for users registered with the Copyright Clearance Center (CCC). The ‘services’ for users can be found on the internet at: www.copyright.com For those organizations that have been granted a photocopy license, a separate system of payment has been arranged.
    [Show full text]
  • An Axiomatic Approach to Block Rewards
    An Axiomatic Approach to Block Rewards Xi Chen Christos Papadimitriou Tim Roughgarden [email protected] [email protected] [email protected] Columbia University Columbia University Columbia University New York, NY 10027 New York, NY 10027 New York, NY 10027 ABSTRACT of view and the methodology of Economic Theory, the science of Proof-of-work blockchains reward each miner for one completed incentives. block by an amount that is, in expectation, proportional to the Flaws in the incentives of a blockchain protocol can manifest number of hashes the miner contributed to the mining of the block. themselves at multiple timescales. For longest-chain proof-of-work Is this proportional allocation rule optimal? And in what sense? And blockchains like Bitcoin, the most well-studied incentive-based what other rules are possible? In particular, what are the desirable attacks, such as selfish mining [4, 5, 10] and transaction sniping [3], properties that any łgoodž allocation rule should satisfy? To answer concern miners reasoning strategically over multiple block creation these questions, we embark on an axiomatic theory of incentives in epochs. For example, in selfish mining, a miner relinquishes revenue proof-of-work blockchains at the time scale of a single block. We in the short term to achieve greater revenue (in expectation) in the consider desirable properties of allocation rules including: symme- long run via a type of forking attack. try; budget balance (weak or strong); sybil-proofness; and various This paper studies incentive issues and potential deviations from grades of collusion-proofness. We show that Bitcoin’s proportional intended miner behavior at the most basic time scale, that of a allocation rule is the unique allocation rule satisfying a certain single block creation epoch.
    [Show full text]
  • Fall 2016 Dear Computer Science Alumni and Friends, These Are
    Alex Aiken Alcatel-Lucent Professor Tencent Chair, Department of Computer Science Fall 2016 Dear Computer Science Alumni and Friends, These are exciting times in computer science, with research, education, and the industry continuing to evolve at a rapid and seemingly increasing rate. The role of computer science is also changing, shifting from being a dynamic, exciting intellectual field driving technological change to becoming a dynamic, exciting intellectual field that is a significant component of many other fields of inquiry as well as a major social and cultural force. The effects of these changes on academic computer science departments, at least in the United States, are well known: soaring enrollments at all levels; a major expansion of research efforts, particularly in collaborations with other fields; and the stresses associated with trying to adapt and grow to meet these challenges and opportunities. The CS department at Stanford has been riding this wave with excitement and enthusiasm—and sometimes exhaustion! In this newsletter, I’ll talk about some of the changes that are taking place. A key part of this process has been discussions with the alumni and friends of the department; many of you have helped by sharing your thoughts with me and other faculty, for which we are sincerely grateful. Faculty Hiring The School of Engineering and the department have agreed that the CS department will grow by a net of 10 full-time faculty positions, from about 40 full-time positions today to about 50. Many of you will no doubt recognize that this won’t change our hiring rate (we’ve been hiring as fast as we can for several years), but it will allow us to take a step back and think about the future composition of the department.
    [Show full text]
  • Letter from the President
    Letter from the President Dear EATCS members, As usual this time of the year, I have the great pleasure to announce the assignments of this year’s Gódel Prize, EATCS Award and Presburger Award. The Gödel Prize 2012, which is co-sponsored by EATCS and ACM SIGACT, has been awarded jointly to Elias Koutsoupias, Christos H. Papadimitriou, Tim Roughgarden, Éva Tardos, Noam Nisan and Amir Ronen. In particular, the prize has been awarded to Elias Koutsoupias and Christos H. Papadimitriou for their paper Worst-case equilibria, Computer Science Review, 3(2): 65-69, 2009; to Tim Roughgarden and Éva Tardos for their paper How Bad Is Selfish Routing? , Journal of the ACM, 49(2): 236-259, 2002; and to Noam Nisan and Amir Ronen for their paper Algorithmic Mechanism Design, Games and Economic Behavior, 35: 166-196, 2001. As you can read in the laudation published in this issue of the bulletin, these three papers contributed highly influential concepts and results that laid the foundation for an explosive growth in algorithmic game theory, a trans-disciplinary combination of the theory of algorithms and the theory of games that has greatly enriched both fields. The purpose of all three papers was to improve our understanding of how the internet and other complex computational systems behave when users and service providers in these systems act selfishly. On behalf of this year’s Gödel Prize Committee (consisting of Sanjeev Arora, Josep Díaz, Giuseppe F. Italiano, Daniel ✸ ❇❊❆❚❈❙ ♥♦ ✶✵✼ ❊❆❚❈❙ ▼❆❚❚❊❘❙ Spielman, Eli Upfal and Mogens Nielsen as chair) and the whole EATCS community I would like to offer our congratulations and deep respect to all of the six winners! The EATCS Award 2012 has been granted to Moshe Vardi for his decisive influence on the development of theoretical computer science, for his pre-eminent career as a distinguished researcher, and for his role as a most illustrious leader and disseminator.
    [Show full text]
  • A Revisionist History of Algorithmic Game Theory
    A Revisionist History of Algorithmic Game Theory Moshe Y. Vardi Rice University Theoretical Computer Science: Vols. A and B van Leeuwen, 1990: Handbook of Theoretical Computer Science Volume A: algorithms and complexity • Volume B: formal models and semantics (“logic”) • E.W. Dijkstra, EWD Note 611: “On the fact that the Atlantic Ocean has two sides” North-American TCS (FOCS&STOC): Volume A. • European TCS (ICALP): Volumes A&B • A Key Theme in FOCS/STOC: Algorithmic Game Theory – algorithm design for strategic environments 1 Birth of AGT: The ”Official” Version NEW YORK, May 16, 2012 – ACM’s Special Interest Group on Algorithms and Computation Theory (SIGACT) together with the European Association for Theoretical Computer Science (EATCS) will recognize three groups of researchers for their contributions to understanding how selfish behavior by users and service providers impacts the behavior of the Internet and other complex computational systems. The papers were presented by Elias Koutsoupias and Christos Papadimitriou, Tim Roughgarden and Eva Tardos, and Noam Nisan and Amir Ronen. They will receive the 2012 Godel¨ Prize, sponsored jointly by SIGACT and EATCS for outstanding papers in theoretical computer science at the International Colloquium on Automata, Languages and Programming (ICALP), July 9–13, in Warwick, UK. 2 Three seminal papers Koutsoupias&Papadimitriou, STACS 1999: Worst- • case Equilibira – introduced the “price of anarchy” concept, a measure of the extent to which competition approximates cooperation, quantifying how much utility is lost due to selfish behaviors on the Internet, which operates without a system designer or monitor striving to achieve the “social optimum.” Roughgarden & Tardos, FOCS 2000: How Bad is • Selfish Routing? – studied the power and depth of the “price of anarchy” concept as it applies to routing traffic in large-scale communications networks to optimize the performance of a congested network.
    [Show full text]
  • Lattice-Inspired Broadcast Encryption and Succinct Ciphertext-Policy ABE
    Lattice-Inspired Broadcast Encryption and Succinct Ciphertext-Policy ABE Zvika Brakerski∗ Vinod Vaikuntanathany Abstract We propose a candidate ciphertext-policy attribute-based encryption (CP-ABE) scheme for circuits, where the ciphertext size depends only on the depth of the policy circuit (and not its size). This, in particular, gives us a Broadcast Encryption (BE) scheme where the size of the keys and ciphertexts have a poly-logarithmic dependence on the number of users. This goal was previously only known to be achievable assuming ideal multilinear maps (Boneh, Waters and Zhandry, Crypto 2014) or indistinguishability obfuscation (Boneh and Zhandry, Crypto 2014) and in a concurrent work from generic bilinear groups and the learning with errors (LWE) assumption (Agrawal and Yamada, Eurocrypt 2020). Our construction relies on techniques from lattice-based (and in particular LWE-based) cryptography. We analyze some attempts at cryptanalysis, but we are unable to provide a security proof. 1 Introduction Broadcast Encryption (BE) [FN93] is an important multi-user generalization of public-key encryp- tion where a broadcaster can send the same message m to an arbitrary subset S ⊆ U, where U is the universe of all the N possible users. A trivial, communication-inefficient, way of achieving this would involve the broadcaster encrypting m separately with the public keys of all users in S, re- sulting in a ciphertext of size O(jSj) (ignoring dependence on the security parameter λ). Broadcast encryption seeks to achieve the same end goal with much better parameters, ideally ciphertexts and keys of size O(λ) (ignoring polylogarithmic factors in jSj and jUj).1 The first solution to the broadcast encryption problem was proposed by Boneh, Gentry and Waters [BGW05] using bilinear maps on elliptic curves.2 Their construction had ciphertexts of size ∗Weizmann Institute of Science.
    [Show full text]
  • Generalization in Adaptive Data Analysis and Holdout Reuse
    Generalization in Adaptive Data Analysis and Holdout Reuse Cynthia Dwork∗ Vitaly Feldmany Moritz Hardtz Toniann Pitassix Omer Reingold{ Aaron Rothk September 28, 2015 Abstract Overfitting is the bane of data analysts, even when data are plentiful. Formal approaches to under- standing this problem focus on statistical inference and generalization of individual analysis procedures. Yet the practice of data analysis is an inherently interactive and adaptive process: new analyses and hypotheses are proposed after seeing the results of previous ones, parameters are tuned on the basis of obtained results, and datasets are shared and reused. An investigation of this gap has recently been initiated by the authors in [DFH+14], where we focused on the problem of estimating expectations of adaptively chosen functions. In this paper, we give a simple and practical method for reusing a holdout (or testing) set to validate the accuracy of hypotheses produced by a learning algorithm operating on a training set. Reusing a holdout set adaptively multiple times can easily lead to overfitting to the holdout set itself. We give an algorithm that enables the validation of a large number of adaptively chosen hypotheses, while provably avoiding overfitting. We illustrate the advantages of our algorithm over the standard use of the holdout set via a simple synthetic experiment. We also formalize and address the general problem of data reuse in adaptive data analysis. We show how the differential-privacy based approach given in [DFH+14] is applicable much more broadly to adaptive data analysis. We then show that a simple approach based on description length can also be used to give guarantees of statistical validity in adaptive settings.
    [Show full text]
  • Computer Science and Game Theory
    review articles 1_CACM_V51.8.indb 74 7/21/08 10:13:35 AM review articles DOI:10.1145/1378704.1378721 University, under the leadership of John The most dramatic interaction between CS von Neumann, in the 1950s.a In this article I try to do two things: and GT may involve game-theory pragmatics. identify the main areas of interaction between computer science and game BY YOAV SHOHAM theory so far; and point to where the most interesting interaction yet may lie—in an area that is still relatively un- derexplored. The first part aims to be an unbiased Computer survey, but it is impossible to avoid bias altogether. Ten researchers survey- ing the interactions between CS and GT would probably write 10 different Science and types of reports. Indeed, several already have (as I will discuss). Moreover, in this brief discussion I cannot possibly do justice to all the work taking place Game Theory in the area. So I try to compensate for these limitations in two ways: I provide a balanced set of initial pointers into the different subareas, without regard to the amount or nature of work that has taken place in each; and I point the reader to other relevant surveys of the CS-GT interaction, each having its own take on things. GAME THEORY HAS influenced many fields, The second part is decidedly subjec- including economics (its initial focus), political tive, but it is still meant to be broadly science, biology, and many others. In recent years, relevant both to computer scientists and game theorists interested in the in- its presence in computer science has become teraction between the disciplines.
    [Show full text]
  • CS364A: Algorithmic Game Theory Lecture #1: Introduction and Examples∗
    CS364A: Algorithmic Game Theory Lecture #1: Introduction and Examples∗ Tim Roughgarden† September 23, 2013 1 Mechanism Design: The Science of Rule-Making This course is roughly organized into three parts, each with its own overarching goal. Here is the first. Course Goal 1 Understand how to design systems with strategic participants that have good performance guarantees. We begin with a cautionary tale. In 2012, the Olympics were held in London. One of the biggest scandals of the event concerned, of all sports, women’s badminton. The scandal did not involve any failed drug tests, but rather a failed tournament design that did not carefully consider incentives. The tournament design that was used is familiar from World Cup soccer. There are four groups (A,B,C,D) of four teams each. The tournament has two phases. In the first ”round-robin” phase, each team plays the other three teams in its group, and does not play teams in other groups. The top two teams from each group advance to the second phase, the bottom two teams from each group are eliminated. In the second phase, the remaining eight teams play a standard ”knockout” tournament (as in tennis, for example): there are four quarterfinals (with the losers eliminated), then two semifinals (with the losers playing an extra match to decide the bronze model), and then the final (the winner gets the gold, the loser the silver). The incentives of participants and of the Olympics committee (and fans) are not neces- sarily aligned in such a tournament. What does a team want? To get as good a medal as possible, of course.
    [Show full text]