LEARNING AND INCENTIVES IN COMPUTER SCIENCE A DISSERTATION SUBMITTED TO THE DEPARTMENT OF COMPUTER SCIENCE AND THE COMMITTEE ON GRADUATE STUDIES OF STANFORD UNIVERSITY IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY Okke Schrijvers June 2017 © 2017 by Okke Joost Schrijvers. All Rights Reserved. Re-distributed by Stanford University under license with the author. This work is licensed under a Creative Commons Attribution- Noncommercial 3.0 United States License. http://creativecommons.org/licenses/by-nc/3.0/us/ This dissertation is online at: http://purl.stanford.edu/jr338fh5605 ii I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Tim Roughgarden, Primary Adviser I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Dan Boneh I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Ashish Goel I certify that I have read this dissertation and that, in my opinion, it is fully adequate in scope and quality as a dissertation for the degree of Doctor of Philosophy. Nicolas Lambert Approved for the Stanford University Committee on Graduate Studies. Patricia J. Gumport, Vice Provost for Graduate Education This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file in University Archives. iii Abstract In this thesis we look at topics in algorithmic game theory, with influences from learning theory. We use tools and insights from game theory to look at areas in computer science, such as machine learning and the bitcoin blockchain, that have been developed incognizant of incentives for selfish behavior. Here we 1) show that there are incentives to behave in a way that’s harmful for the system, 2) give mechanisms where the incentives for the individual and group are aligned, and 3) measure how harmful (having to account for) selfish behavior is. In particular we study the problem of online prediction with expert advice, where we show that when experts are selfish and care about their reputation, the design of incentive compatible algorithms is tightly coupled to strictly proper scoring rules. We give algorithms that have good regret guarantees, and we prove that it is not possible to match regret guarantees for honest experts. For Bitcoin, we show that the way rewards are shared in mining pools can lead to miners strategically hiding work to improve their payout. This holds true even in the absence of outside options for miners, such as other pools or solo mining. We give a novel reward sharing function that does not have perverse incentives, and analyze its performance analytically and through simulations. On the other hand we look at how data can be used in a variety of game theory applications. We ask the questions 1) how can we use data to replace standard informational assumptions in auction theory, 2) how much data do we need for good results in this area, 3) how can we use data to learn about the utilities of agents when we observe behavior, and finally (as misrecorded data may lead algorithms astray) 4) how can we find anomalies in a data set in an unsupervised manner? For the first two questions we look at position auction environments where we give the first computationally efficient algorithm for i.i.d. bidders with irregular value distributions, that achieves revenue arbitrarily close to optimal using polynomial samples. Due to the low sample complexity, our approach leads to a no-regret algorithm in the online setting. To address the third question, we give a computationally efficient algorithm that computes, given a correlated equilibrium (which may be a pure or mixed Nash equilibrium), the set of utilities that are consistent with it. Finally, we give an unsupervised anomaly detection algorithm that runs in a stream, and we show its performance through experiments on real and synthetic data. iv Acknowledgments I am very thankful for the many people who have helped and inspired me during the last five years. First and foremost, Tim Roughgarden has been an incredible academic advisor and co-author on many of the papers that have led to the chapters in this thesis. He has given me the freedom to explore a diverse set of topics, while guiding me in asking the right questions for each of them. Tim has had a profound impact in how I conduct research, and how I communicate my work to others. I’m also very fortunate to have had the opportunity to work with, and learn from, all of my amazing co-authors. In alphabetic order they are: Dan Boneh, Joe Bonneau, Sudipto Guha, Volodymyr Kuleshov, Mohammad Mahdian, Nina Mishra, Tim Roughgarden, Gourav Roy, and Sergei Vassilvitskii.1 The diversity of their expertise and approach to research has truly expanded my experience. Beyond the work I did at Stanford over the past five years, I’ve had the good for- tune to do summer internships at Google, Amazon and Facebook. Not only were these great opportunities to broaden my horizon, the internships have led to publications at KDD and ICML, and a job as core data scientist at Facebook after graduation. Finally, you cannot finish a PhD unless you start a PhD. Bert Heesakkers planted the seed to pursue a Master’s degree and eventually PhD. My professors at Tech- nische Universiteit Eindhoven have been extremely generous in the time they spent on coaching me through the application process and writing letters on my behalf. Finally, I’m grateful for my friends and family, who have borne with me through this entire process. I could not have done this without their support. Okke Schrijvers San Francisco May 31, 2017 1This list includes co-authors from papers that I have not included in the thesis, but where the majority of the work was written during my Ph.D. [Roughgarden and Schrijvers, 2016a], and [Mahdian et al., 2015]. I’m also grateful to my co-authors of [Buchin et al., 2013], [Schrijvers and van Wijk, 2013], and [Schrijvers et al., 2013] which were all published during my Ph.D. but for which the majority of the work was done while I was a Master’s student at Technische Universiteit Eindhoven. v Contents Abstract iv Acknowledgments v 1 Introduction 1 1.1 Research Goals . 3 1.1.1 Goal 1: Understanding Agents Through Data . 3 1.1.2 Goal 2: Robustness to Incentives . 5 1.2 ContributionsoftheThesis . 6 1.2.1 Part I: Learning . 6 1.2.2 Part II: Incentives . 7 I Learning 9 2 Learning Optimal Auctions 10 2.1 Introduction . 10 2.1.1 Our Results . 11 2.1.2 Why Irregular Distributions Are Interesting . 12 2.1.3 WhyIrregularDistributionsAreHard . 13 2.1.4 Related Work . 14 2.2 Preliminaries . 16 2.2.1 The Empirical CDF and the DKW Inequality . 16 2.2.2 Optimal Auctions using the Revenue Curve . 17 2.2.3 Notation.............................. 20 2.3 Additive Loss in Revenue for Single-Item Auctions. 21 2.3.1 TheEmpiricalMyersonAuction . 21 2.3.2 Additive Revenue Loss in Terms of Revenue Curves . 23 2.3.3 Bounding the Error in the Revenue Curve . 24 2.4 Matroid and Position Environments . 30 2.4.1 PositionAuctions. 31 vi 2.4.2 MatroidEnvironments . 31 2.5 No-RegretAlgorithm ........................... 32 2.6 UnboundedDistributions. 33 2.A Reduced Information Model . 36 2.A.1 LowerBound ........................... 36 3 Learning Utilities in Succinct Games 40 3.1 Introduction . 40 3.1.1 Our Contributions. 41 3.1.2 Related Work . 42 3.2 Preliminaries . 43 3.3 Succinct Games . 44 3.3.1 Succinct Representations of Equilibria . 45 3.3.2 WhatitMeanstoRationalizeEquilibria . 45 3.3.3 Non-DegeneracyConditions . 46 3.3.4 The Inverse Game Theory Problem . 46 3.4 LearningUtilitiesinSuccinctGames . 47 3.4.1 General Linear Succinct Games . 47 3.4.2 Inferring Utilities in Popular Succinct Sames . 50 3.5 Learning the Structure of Succinct Games . 53 4 Anomaly Detection in a Stream 56 4.1 Introduction . 57 4.2 DefiningAnomalies ............................ 61 4.3 Forest Maintenance on a Stream . 65 4.3.1 Deletion of Points . 67 4.3.2 InsertionofPoints . .. 68 4.4 Isolation Forest and Other Related Work . 70 4.4.1 TheIsolationForestAlgorithm . 70 4.4.2 Other Related Work . 72 4.5 Experiments . 72 4.5.1 Synthetic Data . 73 4.5.2 RealLifeData:NYCTaxicabs . 74 4.6 Conclusions and Future Work . 76 4.A Proof of Theorem 4.2 . 78 II Incentives 80 5 Online Prediction with Selfish Experts 81 5.1 Introduction . 81 vii 5.1.1 Our Results . 85 5.1.2 Related Work . 86 5.1.3 Organization . 87 5.2 Preliminaries and Model . 88 5.2.1 Standard Model . 88 5.2.2 Selfish Model . 89 5.2.3 Proper Scoring Rules . 90 5.2.4 Online Learning with Quadratic Losses . 91 5.3 DeterministicAlgorithmsforSelfishExperts . 92 5.3.1 Deterministic Online Prediction using a Spherical Rule . 93 5.3.2 TrueLossoftheNon-ICStandardRule. 94 5.4 The Cost of Selfish Experts for IC algorithms . 94 5.4.1 Symmetric Strictly Proper Scoring Rules . 96 5.4.2 Beyond Symmetric Strictly Proper Scoring Rules . 98 5.5 TheCost ofSelfishExpertsforNon-ICAlgorithms . 100 5.6 Randomized Algorithms: Upper and Lower Bounds . 103 5.6.1 ImpossibilityofVanishingRegret .