Applied Math Colloquium

Total Page：16

File Type：pdf, Size：1020Kb

Division of Applied Mathematics Colloquium Thursday, October 10, 2019, 12:00 noon - 1:00pm 170 Hope Street, Room 108 Speaker: Constantinos Daskalakis, MIT Title: Statistical Inference from Dependent Data Abstract: Statistical Learning Theory has largely focused on learning and generalization from independent and identically distributed observations. This assumption is, however, too strong. In many applications, observations are collected on nodes of a network, or some spatial or temporal domain, and are dependent. Examples abound in financial and meteorological applications, and dependencies naturally arise in social networks through peer effects. We study the basic statistical tasks of linear and logistic regression on networked data, where the responses on the nodes of the network are not independent conditioning on the nodes’ vectors of covariates. Given a single observation (across the whole network) from a networked linear or Professor of Computer Science at MIT and a member of CS-AI logistic regression model and under necessary weak dependency Lab. He works on computation theory and its interface with game theory, economics, probability theory, statistics and assumptions, we prove strong consistency results for estimating the model machine learning. He holds a PhD in Computer Science from parameters, recovering the rates achievable in the standard setting with UC-Berkeley. He is the recipient of the Nevanlinna Prize from independent data. We generalize these results beyond linear and logistic the International Mathematical Union, the Kalai Prize from the regression, assuming that the observations satisfy Dobrushin’s condition, Game Theory Society, the ACM Grace Murray Hopper Award, showing how to use Gaussian complexity and VC dimension to control the Simons investigator award, the Bodossaki Foundation Distinguished Young Scientists Award, the SIAM outstanding generalization error. paper prize, the ACM Doctoral Dissertation award.

Copy Link

Recommended publications

Constantinos Daskalakis
The Work of Constantinos Daskalakis Allyn Jackson How long does it take to solve a problem on a computer? This seemingly innocuous question, unanswerable in general, lies at the heart of computa- tional complexity theory. With deep roots in mathematics and logic, this area of research seeks to understand the boundary between which problems are efficiently solvable by computer and which are not. Computational complexity theory is one of the most vibrant and inven- tive branches of computer science, and Constantinos Daskalakis stands out as one of its brightest young lights. His work has transformed understand- ing about the computational complexity of a host of fundamental problems in game theory, markets, auctions, and other economic structures. A wide- ranging curiosity, technical ingenuity, and deep theoretical insight have en- abled Daskalakis to bring profound new perspectives on these problems. His first outstanding achievement grew out of his doctoral dissertation and lies at the border of game theory and computer science. Game the- ory models interactions of rational agents|these could be individual people, businesses, governments, etc.|in situations of conflict, competition, or coop- eration. A foundational concept in game theory is that of Nash equilibrium. A game reaches a Nash equilibrium if each agent adopts the best strategy possible, taking into account the strategies of the other agents, so that there is no incentive for any agent to change strategy. The concept is named after mathematician John F. Nash, who proved in 1950 that all games have Nash equilibria. This result had a profound effect on economics and earned Nash the Nobel Prize in Economic Sciences in 1994.
[Show full text]
More Revenue from Two Samples Via Factor Revealing Sdps
EC’20 Session 2e: Revenue Maximization More Revenue from Two Samples via Factor Revealing SDPs CONSTANTINOS DASKALAKIS, Massachusetts Institute of Technology MANOLIS ZAMPETAKIS, Massachusetts Institute of Technology We consider the classical problem of selling a single item to a single bidder whose value for the item is drawn from a regular distribution F, in a “data-poor” regime where F is not known to the seller, and very few samples from F are available. Prior work [9] has shown that one sample from F can be used to attain a 1/2-factor approximation to the optimal revenue, but it has been challenging to improve this guarantee when more samples from F are provided, even when two samples from F are provided. In this case, the best approximation known to date is 0:509, achieved by the Empirical Revenue Maximizing (ERM) mechanism [2]. We improve this guarantee to 0:558, and provide a lower bound of 0:65. Our results are based on a general framework, based on factor-revealing Semidefnite Programming relaxations aiming to capture as tight as possible a superset of product measures of regular distributions, the challenge being that neither regularity constraints nor product measures are convex constraints. The framework is general and can be applied in more abstract settings to evaluate the performance of a policy chosen using independent samples from a distribution and applied on a fresh sample from that same distribution. CCS Concepts: • Theory of computation → Algorithmic game theory and mechanism design; Algo- rithmic mechanism design. Additional Key Words and Phrases: revenue maximization, two samples, mechanism design, semidefnite relaxation ACM Reference Format: Constantinos Daskalakis and Manolis Zampetakis.
[Show full text]
Testing Ising Models
Testing Ising Models The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters. Citation Daskalakis, Constantinos et al. “Testing Ising Models.” IEEE Transactions on Information Theory, 65, 11 (November 2019): 6829 - 6852 © 2019 The Author(s) As Published 10.1109/TIT.2019.2932255 Publisher Institute of Electrical and Electronics Engineers (IEEE) Version Author's final manuscript Citable link https://hdl.handle.net/1721.1/129990 Terms of Use Creative Commons Attribution-Noncommercial-Share Alike Detailed Terms http://creativecommons.org/licenses/by-nc-sa/4.0/ Testing Ising Models∗ § Constantinos Daskalakis† Nishanth Dikkala‡ Gautam Kamath July 12, 2019 Abstract Given samples from an unknown multivariate distribution p, is it possible to distinguish whether p is the product of its marginals versus p being far from every product distribution? Similarly, is it possible to distinguish whether p equals a given distribution q versus p and q being far from each other? These problems of testing independence and goodness-of-fit have received enormous attention in statistics, information theory, and theoretical computer science, with sample-optimal algorithms known in several interesting regimes of parameters. Unfortu- nately, it has also been understood that these problems become intractable in large dimensions, necessitating exponential sample complexity. Motivated by the exponential lower bounds for general distributions as well as the ubiquity of Markov Random Fields (MRFs) in the modeling of high-dimensional distributions, we initiate the study of distribution testing on structured multivariate distributions, and in particular the prototypical example of MRFs: the Ising Model. We demonstrate that, in this structured setting, we can avoid the curse of dimensionality, obtaining sample and time efficient testers for independence and goodness-of-fit.
[Show full text]
The Complexity of Nash Equilibria by Constantinos Daskalakis Diploma
The Complexity of Nash Equilibria by Constantinos Daskalakis Diploma (National Technical University of Athens) 2004 A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Christos H. Papadimitriou, Chair Professor Alistair J. Sinclair Professor Satish Rao Professor Ilan Adler Fall 2008 The dissertation of Constantinos Daskalakis is approved: Chair Date Date Date Date University of California, Berkeley Fall 2008 The Complexity of Nash Equilibria Copyright 2008 by Constantinos Daskalakis Abstract The Complexity of Nash Equilibria by Constantinos Daskalakis Doctor of Philosophy in Computer Science University of California, Berkeley Professor Christos H. Papadimitriou, Chair The Internet owes much of its complexity to the large number of entities that run it and use it. These entities have different and potentially conflicting interests, so their interactions are strategic in nature. Therefore, to understand these interactions, concepts from Economics and, most importantly, Game Theory are necessary. An important such concept is the notion of Nash equilibrium, which provides us with a rigorous way of predicting the behavior of strategic agents in situations of conflict. But the credibility of the Nash equilibrium as a framework for behavior-prediction depends on whether such equilibria are efficiently computable. After all, why should we expect a group of rational agents to behave in a fashion that requires exponential time to be computed? Motivated by this question, we study the computational complexity of the Nash equilibrium. We show that computing a Nash equilibrium is an intractable problem.
[Show full text]
On the Cryptographic Hardness of Finding a Nash Equilibrium
On the Cryptographic Hardness of Finding a Nash Equilibrium Nir Bitansky∗ Omer Panethy Alon Rosenz August 14, 2015 Abstract We prove that finding a Nash equilibrium of a game is hard, assuming the existence of indistin- guishability obfuscation and one-way functions with sub-exponential hardness. We do so by showing how these cryptographic primitives give rise to a hard computational problem that lies in the complexity class PPAD, for which finding Nash equilibrium is complete. Previous proposals for basing PPAD-hardness on program obfuscation considered a strong “virtual black-box” notion that is subject to severe limitations and is unlikely to be realizable for the programs in question. In contrast, for indistinguishability obfuscation no such limitations are known, and recently, several candidate constructions of indistinguishability obfuscation were suggested based on different hardness assumptions on multilinear maps. Our result provides further evidence of the intractability of finding a Nash equilibrium, one that is extrinsic to the evidence presented so far. ∗MIT. Email: [email protected]. yBoston University. Email: [email protected]. Supported by the Simons award for graduate students in theoretical computer science and an NSF Algorithmic foundations grant 1218461. zEfi Arazi School of Computer Science, IDC Herzliya, Israel. Email: [email protected]. Supported by ISF grant no. 1255/12 and by the ERC under the EU’s Seventh Framework Programme (FP/2007-2013) ERC Grant Agreement n. 307952. 1 Introduction The notion of Nash equilibrium is fundamental to game theory. While a mixed Nash equilibrium is guar- anteed to exist in any game [Nas51], there is no known polynomial-time algorithm for finding one.
[Show full text]
SIGACT FY'06 Annual Report July 2005
SIGACT FY’06 Annual Report July 2005 - June 2006 Submitted by: Richard E. Ladner, SIGACT Chair 1. Awards Gödel Prize: Manindra Agrawal, Neeraj Kayal, and Nitin Saxena for their paper "PRIMES is in P" in the Annals of Mathematics 160, 1-13, 2004. The Gödel Prize is awarded jointly by SIGACT and EATCS. Donald E. Knuth Prize: Mihalis Yannakakis for his seminal and prolific research in computational complexity theory, database theory, computer-aided verification and testing, and algorithmic graph theory. The Knuth Prize is given jointly by SIGACT and IEEE Computer Society TCMFC. Paris Kanellakis Theory and Practice Award: Gerard J. Holzmann, Robert P. Kurshan, Moshe Y. Vardi, and Pierre Wolper recognizing their development of automata-theoretic techniques for reactive-systems verification, and the practical realization of powerful formal-verification tools based on these techniques. This award is an ACM award sponsored in part by SIGACT. Edsger W. Dijkstra Prize in Distributed Computing: Marshal Pease, Robert Shostak, and Leslie Lamport for their paper “Reaching agreement in the presence of faults” in Journal of the ACM 27 (2): 228-234, 1980. The Dijkstra Prize is given jointly by SIGACT and SIGOPS. ACM-SIGACT Distinguished Service Award: Tom Leighton for his exemplary work as Chair of SIGACT, Editorships, Program Chairships and the establishment of permanent endowments for SIGACT awards. Best Paper Award at STOC 2006: Irit Dinur for “The PCP Theorem via Gap Amplification”. SIGACT Danny Lewin Best Student Paper Award at STOC 2006: Anup Rao and Jakob Nordstrom for “Extractors for a Constant Number of Polynomial Min-Entropy Independent Sources” (Anup Rao) and “Narrow Proofs May Be Spacious: Separating Space and Width in Resolution” (Jakob Nordstrom).
[Show full text]
The Complexity of Constrained Min-Max Optimization
The Complexity of Constrained Min-Max Optimization Constantinos Daskalakis Stratis Skoulakis Manolis Zampetakis MIT SUTD MIT [email protected] [email protected] [email protected] September 22, 2020 Abstract Despite its important applications in Machine Learning, min-max optimization of objective functions that are nonconvex-nonconcave remains elusive. Not only are there no known first- order methods converging even to approximate local min-max points, but the computational complexity of identifying them is also poorly understood. In this paper, we provide a charac- terization of the computational complexity of the problem, as well as of the limitations of first- order methods in constrained min-max optimization problems with nonconvex-nonconcave objectives and linear constraints. As a warm-up, we show that, even when the objective is a Lipschitz and smooth differ- entiable function, deciding whether a min-max point exists, in fact even deciding whether an approximate min-max point exists, is NP-hard. More importantly, we show that an approxi- mate local min-max point of large enough approximation is guaranteed to exist, but finding one such point is PPAD-complete. The same is true of computing an approximate fixed point of the (Projected) Gradient Descent/Ascent update dynamics. An important byproduct of our proof is to establish an unconditional hardness result in the Nemirovsky-Yudin [NY83] oracle optimization model. We show that, given oracle access to some function f : P! [−1, 1] and its gradient r f , where P ⊆ [0, 1]d is a known convex polytope, every algorithm that finds a #-approximate local min-max point needs to make a number of queries that is exponential in at least one of 1/#, L, G, or d, where L and G are respectively the smoothness and Lipschitzness of f and d is the dimension.
[Show full text]
Arxiv:1908.02894V4 [Cs.LG] 25 Apr 2021 Programming Algorithms, and Selling Mechanisms
How much data is sufficient to learn high-performing algorithms? Generalization guarantees for data-driven algorithm design Maria-Florina Balcan Dan DeBlasio Travis Dick Carnegie Mellon University University of Texas at El Paso University of Pennsylvania [email protected] [email protected] [email protected] Carl Kingsford Tuomas Sandholm Ellen Vitercik Carnegie Mellon University Carnegie Mellon University Carnegie Mellon University Ocean Genomics, Inc. Optimized Markets, Inc. [email protected] [email protected] Strategic Machine, Inc. Strategy Robot, Inc. [email protected] April 27, 2021 Abstract Algorithms often have tunable parameters that impact performance metrics such as run- time and solution quality. For many algorithms used in practice, no parameter settings admit meaningful worst-case bounds, so the parameters are made available for the user to tune. Al- ternatively, parameters may be tuned implicitly within the proof of a worst-case approximation ratio or runtime bound. Worst-case instances, however, may be rare or nonexistent in practice. A growing body of research has demonstrated that data-driven algorithm design can lead to sig- nificant improvements in performance. This approach uses a training set of problem instances sampled from an unknown, application-specific distribution and returns a parameter setting with strong average performance on the training set. We provide a broadly applicable theory for deriving generalization guarantees that bound the difference between the algorithm's average performance over the training set and its expected performance on the unknown distribution. Our results apply no matter how the parameters are tuned, be it via an automated or manual approach.
[Show full text]
Multiplicative Incentive Mechanisms for Crowdsourcing
Journal of Machine Learning Research 17 (2016) 1-52 Submitted 12/15; Revised 7/16; Published 9/16 Double or Nothing: Multiplicative Incentive Mechanisms for Crowdsourcing Nihar B. Shah [email protected] Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, CA 94720 USA Dengyong Zhou [email protected] Machine Learning Department Microsoft Research One Microsoft Way, Redmond 98052 USA Editor: Qiang Liu Abstract Crowdsourcing has gained immense popularity in machine learning applications for obtaining large amounts of labeled data. Crowdsourcing is cheap and fast, but suffers from the problem of low- quality data. To address this fundamental challenge in crowdsourcing, we propose a simple pay- ment mechanism to incentivize workers to answer only the questions that they are sure of and skip the rest. We show that surprisingly, under a mild and natural “no-free-lunch” requirement, this mechanism is the one and only incentive-compatible payment mechanism possible. We also show that among all possible incentive-compatible mechanisms (that may or may not satisfy no-free- lunch), our mechanism makes the smallest possible payment to spammers. We further extend our results to a more general setting in which workers are required to provide a quantized confidence for each question. Interestingly, this unique mechanism takes a “multiplicative” form. The simplicity of the mechanism is an added benefit. In preliminary experiments involving over 900 worker-task pairs, we observe a significant drop in the error rates under this unique mechanism for the same or lower monetary expenditure. Keywords: high-quality labels, supervised learning, crowdsourcing, mechanism design, proper scoring rules 1.
[Show full text]
The 2011 Noticesindex
The 2011 Notices Index January issue: 1–224 May issue: 649–760 October issue: 1217–1400 February issue: 225–360 June/July issue: 761–888 November issue: 1401–1528 March issue: 361–520 August issue: 889–1048 December issue: 1529–1672 April issue: 521–648 September issue: 1049–1216 2011 Morgan Prize, 599 Index Section Page Number 2011 Report of the Executive Director, State of the AMS, 1006 The AMS 1652 2011 Satter Prize, 601 Announcements 1653 2011 Steele Prizes, 593 Articles 1653 2011–2012 AMS Centennial Fellowship Awarded, 835 Authors 1655 American Mathematical Society Centennial Fellowships, Deaths of Members of the Society 1657 1135 Grants, Fellowships, Opportunities 1659 American Mathematical Society—Contributions, 723 Letters to the Editor 1661 Meetings Information 1661 AMS-AAAS Mass Media Summer Fellowships, 1303 New Publications Offered by the AMS 1662 AMS Announces Congressional Fellow, 843 Obituaries 1662 AMS Announces New Blog for Job Seekers, 1603 Opinion 1662 AMS Congressional Fellowship, 1467 Prizes and Awards 1662 AMS Department Chairs Workshop, 1468 Prizewinners 1665 AMS Epsilon Fund, 1466 Reference and Book List 1668 AMS Holds Workshop for Department Chairs, 844 Reviews 1668 AMS Announces Mass Media Fellowship Award, 843 Surveys 1668 AMS Centennial Fellowships, Invitation for Applications Tables of Contents 1668 for Awards, 1466 AMS Email Support for Frequently Asked Questions, 326 AMS Holds Workshop for Department Chairs, 730 About the Cover AMS Hosts Congressional Briefing, 73 35, 323, 475, 567, 717, 819, 928, 1200, 1262,
[Show full text]
A Poet of Computation Who Uncovers Distant Truths
Quanta Magazine A Poet of Computation Who Uncovers Distant Truths The theoretical computer scientist Constantinos Daskalakis has won the Rolf Nevanlinna Prize for explicating core questions in game theory and machine learning. By Erica Klarreich Photo by Cassandra Klos for Quanta Magazine; Illustration by Olena Shmahalo/Quanta Magazine Constantinos Daskalakis on the banks of the Charles River in Cambridge, Massachusetts. Scroll down to the bottom of Constantinos Daskalakis’ web page — past links to his theoretical computer science papers and his doctoral students at the Massachusetts Institute of Technology — and you will come upon a spare, 21-line poem by Constantine Cavafy, “The Satrapy.” Written in 1910, it addresses an unnamed individual who is “made for fine and great works” but https://www.quantamagazine.org/computer-scientist-constantinos-daskalakis-wins-nevanlinna-prize-20180801/ August 1, 2018 Quanta Magazine who, having met with small-mindedness and indifference, gives up on his dreams and goes to the court of the Persian king Artaxerxes. The king lavishes satrapies (provincial governorships) upon him, but his soul, Cavafy writes, “weeps for other things … the hard-won and inestimable Well Done; the Agora, the Theater, and the Laurels” — all the things Artaxerxes cannot give him. “Where will you find these in a satrapy,” Cavafy asks, “and what life can you live without these.” For Daskalakis, the poem serves as a sort of talisman, to guard him against base motives. “It’s a moral compass, if you want,” he said. “I want to have this constant reminder that there are some noble ideas that you’re serving, and don’t forget that when you make decisions.” The decisions the 37-year-old Daskalakis has made over the course of his career — such as forgoing a lucrative job right out of college and pursuing the hardest problems in his field — have all been in the service of uncovering distant truths.
[Show full text]
Provably Effective Algorithms for Min-Max Optimization by Qi
Provably Effective Algorithms for Min-Max Optimization by Qi Lei DISSERTATION Presented to the Faculty of the Graduate School of The University of Texas at Austin in Partial Fulﬁllment of the Requirements for the Degree of DOCTOR OF PHILOSOPHY THE UNIVERSITY OF TEXAS AT AUSTIN May 2020 Acknowledgments I would like to give the most sincere thanks to my advisors Alexandros G. Dimakis and Inderjit S. Dhillon. Throughout my PhD, they constantly inﬂuence me with their splendid research taste, vision, and attitude. Their enthusiasm in the pursuit of challenging and fundamental machine learning problems has inspired me to do so as well. They encouraged me to continue the academic work and they have set as role models of being a professor with an open mind in discussions, criticisms, and collaborations. I was fortunate in relatively big research groups throughout my stay at UT, where I was initially involved in Prof. Dhillon’s group and later more in Prof. Dimakis’ group. Both groups consist of good researchers and endow me with a collaborative research environment. The weekly group meetings push me to study different research directions and ideas on a daily basis. I am grateful to all my colleagues, especially my collaborators among them: Kai Zhong, Jiong Zhang, Ian Yen, Prof. Cho-Jui Hsieh, Hsiang-Fu Yu, Chao-Yuan Wu, Ajil Jalal, Jay Whang, Rashish Tandon, Shanshan Wu, Matt Jordan, Sriram Ravula, Erik Lindgren, Dave Van Veen, Si Si, Kai-Yang Chiang, Donghyuk Shin, David Inouye, Nikhil Rao, and Joyce Whang. I am also fortunate enough to collaborate with some other professors from UT like Prof.
[Show full text]