DOCSLIB.ORG

Theory and Algorithms for Modern Problems in Machine Learning and an Analysis of Markets

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Theory and Algorithms for Modern Problems in Machine Learning and an Analysis of Markets Theory and Algorithms for Modern Problems in Machine Learning and an Analysis of Markets by Ashish Rastogi A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computer Science Courant Institute of Mathematical Sciences New York University May 2008 Richard Cole—Advisor Mehryar Mohri—Advisor °c Ashish Rastogi All Rights Reserved, 2008 To the most wonderful parents in the whole world, Mrs. Asha Rastogi and Mr. Shyam Lal Rastogi iv Acknowledgements First and foremost, I would like to thank my advisors, Professor Richard Cole and Professor Mehryar Mohri, for their unwavering support, guidance and constant encouragement. They have been inspiring mentors and much of what lies in the following pages can be credited to them. Working under their supervision has been one of the most enriching experiences of my life. I would also like to thank Professor Joel Spencer, Professor Arun Sun- dararajan, Professor Subhash Khot and Dr. Corinna Cortes for agreeing to serve as members on my thesis committee. Professor Spencer’s class on Random Graphs remains one of the most stim- ulating courses I undertook as a graduate student. Internships at Google through the summers of 2005, 2006 and 2007 were some of the most enjoy- able periods of my graduate school life. Many thanks are due to Dr. Corinna Cortes for providing me with the opportunity to work on several challenging problems at Google. Research initiated during these internships culminated in the development of ideas that form the bulk of this thesis. I would also like to thank my peers from the graduate school. Spirited discussions in the reading group meetings and during various seminars were directly responsible for the development of a “research attitude”, a critical ingredient for the successful completion of graduate studies. For all this, I thank Eugene Weinstein and Tyler Neylon. v Pursuing a Ph.D. can be an arduous affair. It entails everything from hav- ing to cope with failure to the exhilarating feeling of settling a research problem successfully. I would like to express my gratitude towards my friends, espe- cially Shruti Haldea, Ritu Gupta, Abhimanyu Yadav, Usha Mallya, Hameer Ruparel, Chris Wu and Marjorie Levy for providing an excellent web of sup- port. Finally, none of this would ever have transpired without the unselfish love and support of my precious parents and my delightful sister Ragini. Their constant support, patient love and care and unswerving faith has gotten me where I am. This thesis is dedicated to them. vi Abstract The unprecedented growth of the Internet over the past decade and of data collection, more generally, has given rise to vast quantities of digital in- formation, ranging from web documents and images, genomic databases to a vast array of business customer information. Consequently, it is of growing importance to develop tools and models that enable us to better understand this data and to design data-driven algorithms that leverage this information. This thesis provides several fundamental theoretical and algorithmic results for tackling such problems with applications to speech recognition, image pro- cessing, natural language processing, computational biology and web-based algorithms. • Probabilistic automata provide an efficient and compact way to model sequence-oriented data such as speech or web documents. Measuring the similarity of such automata provides a way of comparing the objects they model, and is an essential first step in organizing this type of data. We present algorithmic and hardness results for computing various discrep- ancies (or dissimilarities) between probabilistic automata, including the relative entropy and the Lp distance; we also give an efficient algorithm to determine if two probabilistic automata are equivalent. In addition, we study the complexity of computing the norms of probabilistic automata. • Widespread success of search engines and information retrieval systems has led to large scale collection of rating information which is being used vii to provide personalized rankings. We examine an alternate formulation of the ranking problem for search engines motivated by the requirement that in addition to accurately predicting pairwise ordering, ranking sys- tems must also preserve the magnitude of the preferences or the dif- ference between ratings. We present algorithms with sound theoretical properties, and verify their efficacy through experiments. • Organizing and querying large amounts of digitized data such as images and videos is a challenging task because little or no label information is available. This motivates transduction, a setting in which the learning algorithm can leverage unlabeled data during training to improve per- formance. We present novel error bounds for a family of transductive regression algorithms and validate their usefulness through experiments. • Finally, price discovery in a market setting can be viewed as an (ongoing) learning problem. Specifically, the problem is to find and maintain a set of prices that balance supply and demand, a core topic in economics. This appears to involve complex implicit and possibly large-scale infor- mation transfers. We show that finding equilibrium prices, even approx- imately, in discrete markets is NP-hard and complement the hardness result with a matching polynomial time approximation algorithm. We also give a new way of measuring the quality of an approximation to equilibrium prices that is based on a natural aggregation of the dissatis- faction of individual market participants. viii Contents Dedication . iv Acknowledgements . v Abstract . vii 1 Distances Between Probabilistic Automata 1 1.1 Introduction . 1 1.2 Preliminaries . 6 1.2.1 Semirings . 6 1.2.2 Probabilistic Automata and Shortest-Distances . 9 1.2.3 Algorithms for Computing Shortest-Distances . 14 1.2.4 Composition of Weighted Automata . 19 1.2.5 Distances Between Distributions And Norms . 20 1.2.6 Problems Studied . 22 1.3 Algorithms for Computation of Distances . 22 1.3.1 Relative Entropy of Unambiguous Automata . 23 1.3.2 L2p Distance . 36 1.3.3 Hellinger Distance . 40 ix 1.4 Hardness Results . 41 1.4.1 L2p+1 Distance and L∞ Distance . 41 1.4.2 Relative Entropy of Arbitrary Automata . 55 1.5 Equivalence of Probabilistic Automata . 59 1.6 Relative Entropy As a Kernel . 61 1.7 Computation of the Norm . 64 1.7.1 Norm of an Unambiguous Automaton . 65 1.7.2 Norm of Arbitrary Automata . 67 1.7.3 Approximate Computation of Lp-norm . 67 1.7.4 Approximate Computation of Entropy . 68 1.8 Conclusion . 71 2 Magnitude-Preserving Ranking Algorithms 74 2.1 Introduction and Motivation . 74 2.2 Preliminaries . 78 2.2.1 Formulation of the Problem . 78 2.2.2 Cost Functions . 79 2.2.3 Kernels and Regularization . 81 2.3 Algorithms . 84 2.3.1 Objective Functions . 84 2.3.2 MPRank . 85 2.3.3 SVRank . 88 2.3.4 On-line Version of MPRank . 90 x 2.3.5 Leave-One-Out Analysis for MPRank . 92 2.4 Stability bounds . 98 2.4.1 Magnitude-preserving regularization algorithms . 99 2.5 Experiments . 106 2.5.1 MovieLens Dataset . 108 2.5.2 Jester Joke Dataset . 109 2.5.3 Netflix Dataset . 109 2.5.4 Book-Crossing Dataset . 110 2.5.5 Performance Measures and Results . 110 2.5.6 On-line Version of MPRank . 113 2.6 Conclusion . 114 3 Transductive Regression 116 3.1 Introduction . 116 3.2 Preliminaries . 122 3.2.1 Learning Setting . 122 3.2.2 Transductive Stability . 123 3.3 Transductive Regression Stability Bounds . 125 3.3.1 Bound for Sampling without Replacement . 125 3.3.2 Transductive Stability Bound . 129 3.4 Stability of Local Transductive Regression Algorithms . 132 3.4.1 Local Transductive Regression Algorithms . 133 3.5 Stability Based on Closed-Form Solution . 141 xi 3.5.1 Unconstrained Regularization Algorithms . 141 3.5.2 Stability of Constrained Regularization Algorithms . 146 3.5.3 Making Seemingly Unstable Algorithms Stable . 148 3.6 Experiments . 149 3.6.1 Model Selection Based on Bound . 149 3.7 Conclusion . 155 4 An Analysis of Discrete Markets 156 4.1 Introduction . 156 4.2 Definitions . 171 4.3 Hardness of Computing Near-Equilibrium Prices . 178 4.3.1 The Balanced Max-3SAT-3 problem . 184 4.3.2 Details of the Hardness Construction . 186 4.4 A Matching Algorithm . 207 4.5 A Local Tatonnement Algorithm . 213 4.5.1 Introduction and Motivation . 213 4.5.2 Analysis of Convergence . 218 4.6 Relationship Between Discontent and ²-closeness in Utility . 238 Bibliography . 240 xii Chapter 1 Distances Between Probabilistic Automata 1.1 Introduction A probabilistic automaton is a finite automaton in which each transition carries a non-negative weight. The weight associated with a path π in the automaton is the product of the weights of the transitions that appear on π and the probability associated with a certain string x accepted by the automaton is the sum of the weights of all the paths on which x is accepted. In addition, the sum of the weights of all strings accepted by the automaton is one. Thus, such an automaton represents a probability distribution over a regular set. Probabilistic automata were introduced in [81] and are extensively used in a variety of areas of computer science including text and speech process- 1 ing [73], image processing [39], and computational biology [43]. In natural language-processing applications, for example, probabilistic automata are used to describe morphological and phonological rules [61]. Closely related to probabilistic automata are Hidden Markov Models, which are also used extensively in statistical learning [42, 82]. In a Hidden Markov Model, the weight associated with a transition can itself be a random variable whose value is drawn from some distribution. However, when each transi- tion has a fixed constant weight, a Hidden Markov Model is identical to a probabilistic automaton.
Load more

Recommended publications
  • Typical Stability
    Typical Stability Raef Bassily∗ Yoav Freundy Abstract In this paper, we introduce a notion of algorithmic stability called typical stability. When our goal is to release real-valued queries (statistics) computed over a dataset, this notion does not require the queries to be of bounded sensitivity – a condition that is generally assumed under differential privacy [DMNS06, Dwo06] when used as a notion of algorithmic stability [DFH+15b, DFH+15c, BNS+16] – nor does it require the samples in the dataset to be independent – a condition that is usually assumed when generalization-error guarantees are sought. Instead, typical stability requires the output of the query, when computed on a dataset drawn from the underlying distribution, to be concentrated around its expected value with respect to that distribution. Typical stability can also be motivated as an alternative definition for database privacy. Like differential privacy, this notion enjoys several important properties including robustness to post-processing and adaptive composition. However, privacy is guaranteed only for a given family of distributions over the dataset. We also discuss the implications of typical stability on the generalization error (i.e., the difference between the value of the query computed on the dataset and the expected value of the query with respect to the true data distribution). We show that typical stability can control generalization error in adaptive data analysis even when the samples in the dataset are not necessarily independent and when queries to be computed are not necessarily of bounded- sensitivity as long as the results of the queries over the dataset (i.e., the computed statistics) follow a distribution with a “light” tail.
    [Show full text]
  • An Efficient Boosting Algorithm for Combining Preferences Raj
    An Efficient Boosting Algorithm for Combining Preferences by Raj Dharmarajan Iyer Jr. Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Master of Science at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY September 1999 © Massachusetts Institute of Technology 1999. All rights reserved. A uth or .............. ..... ..... .................................. Department of E'ectrical ngineering and Computer Science August 24, 1999 C ertified by .. ................. ...... .. .............. David R. Karger Associate Professor Thesis Supervisor Accepted by............... ........ Arthur C. Smith Chairman, Departmental Committe Graduate Students MACHU OF TEC Lo NOV LIBRARIES An Efficient Boosting Algorithm for Combining Preferences by Raj Dharmarajan Iyer Jr. Submitted to the Department of Electrical Engineering and Computer Science on August 24, 1999, in partial fulfillment of the requirements for the degree of Master of Science Abstract The problem of combining preferences arises in several applications, such as combining the results of different search engines. This work describes an efficient algorithm for combining multiple preferences. We first give a formal framework for the problem. We then describe and analyze a new boosting algorithm for combining preferences called RankBoost. We also describe an efficient implementation of the algorithm for certain natural cases. We discuss two experiments we carried out to assess the performance of RankBoost. In the first experi- ment, we used the algorithm to combine different WWW search strategies, each of which is a query expansion for a given domain. For this task, we compare the performance of Rank- Boost to the individual search strategies. The second experiment is a collaborative-filtering task for making movie recommendations.
    [Show full text]
  • LNAI 4264, Pp
    Lecture Notes in Artificial Intelligence 4264 Edited by J. G. Carbonell and J. Siekmann Subseries of Lecture Notes in Computer Science José L. Balcázar Philip M. Long Frank Stephan (Eds.) Algorithmic Learning Theory 17th International Conference, ALT 2006 Barcelona, Spain, October 7-10, 2006 Proceedings 1 3 Series Editors Jaime G. Carbonell, Carnegie Mellon University, Pittsburgh, PA, USA Jörg Siekmann, University of Saarland, Saarbrücken, Germany Volume Editors José L. Balcázar Universitat Politecnica de Catalunya, Dept. Llenguatges i Sistemes Informatics c/ Jordi Girona, 1-3, 08034 Barcelona, Spain E-mail: [email protected] Philip M. Long Google 1600 Amphitheatre Parkway, Mountain View, CA 94043, USA E-mail: [email protected] Frank Stephan National University of Singapore, Depts. of Mathematics and Computer Science 2 Science Drive 2, Singapore 117543, Singapore E-mail: [email protected] Library of Congress Control Number: 2006933733 CR Subject Classification (1998): I.2.6, I.2.3, F.1, F.2, F.4, I.7 LNCS Sublibrary: SL 7 – Artificial Intelligence ISSN 0302-9743 ISBN-10 3-540-46649-5 Springer Berlin Heidelberg New York ISBN-13 978-3-540-46649-9 Springer Berlin Heidelberg New York This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer.
    [Show full text]
  • MMLS 2017 Booklet
    MMLS 2017 Booklet Monday, June 19 9:30 - 11:15 TTIC Continental breakfast. (TTIC Colloquium: 10:00-11:00.) 11:15 - 11:30 GPAH Opening remarks: Po-Ling Loh. 11:30 - 12:20 GPAH Plenary speaker: Devavrat Shah . (Chair: Po-Ling Loh .) Latent Variable Model Estimation via Collaborative Filtering. 12:20 - 2:50 GPAH Lunch, posters. 2:50 - 3:30 GPAH Invited talks (chair: Mesrob Ohannessian ). 2:50: Rina Foygel Projected Gradient Descent with Nonconvex Constraints. 3:10: Maxim Raginsky Non-Convex Learning via Stochastic Gradient Langevin Dynamics. 3:30 - 4:00 GPAH Coffee Break. 4:00 - 4:50 GPAH Plenary speaker: Rayid Ghani . (Chair: Nati Srebro .) Machine Learning for Public Policy: Opportunities and Challenges 4:50 - 5:30 GPAH Invited talks (chair: Laura Balzano ). 4:50: Dimitris Papailiopoulos Gradient Diversity Empowers Distributed Learning. 5:10: Alan Ritter Large-Scale Learning for Information Extraction. 5:45 - 7:00 TTIC Reception, with remarks by Sadaoki Furui (TTIC President). Tuesday, June 20 8:30 - 9:50 GPAH Continental breakfast. 9:00 - 9:50 GPAH Bonus speaker: Larry Wasserman . (Chair: Mladen Kolar .) Locally Optimal Testing. [ Cancelled. ] 9:50 - 10:50 GPAH Invited talks (chair: Misha Belkin ). 9:50: Srinadh Bhojanapalli Effectiveness of Local Search for Low Rank Recovery 10:10: Niao He Learning From Conditional Distributions. 10:30: Clayton Scott Nonparametric Preference Completion. 10:50 - 11:20 GPAH Coffee break. 11:20 - 12:20 GPAH Invited talks (chair: Jason Lee ). 11:20: Lev Reyzin On the Complexity of Learning from Label Proportions. 11:40: Ambuj Tewari Random Perturbations in Online Learning. 12:00: Risi Kondor Multiresolution Matrix Factorization.
    [Show full text]
  • Nil Ib N O Ir Ali Mi S Na El Oo B Ilp Itl
    ecneicS retupmoC retupmoC ecneicS ecneicS - o t- o l t aA l aA DD DD 9 9 / / OOnn BBiilliinneeaarr TTeecchhnniiqquueess ffoorr 0202 0202 a p pa p r p a r K a K ii t t t aM t aM SSiimmiillaarriittyy SSeeaarrcchh aanndd BBoooolleeaann MMaattrriixx M ultiplication Multiplication MMaatttit iK Kaarprpppaa noi taci lpi t luM xi r taM naelooB dna hcraeS yt i ral imiS rof seuqinhceT raeni l iB nO noi taci lpi t luM xi r taM naelooB dna hcraeS yt i ral imiS rof seuqinhceT raeni l iB nO BBUUSSININESESS + + ECECOONNOOMMY Y NSI I NBS NBS 879 - 879 - 259 - 259 - 06 - 06 - 5198 - 5198 7 - 7 ( p ( r p i n r t i n t de ) de ) AARRT T+ + NSI I NBS NBS 879 - 879 - 259 - 259 - 06 - 06 - 6198 - 6198 4 - 4 ( ( dp f dp ) f ) DDESESIGIGNN + + NSI I NSS NSS 9971 - 9971 - 4394 4394 ( p ( r p i n r t i n t de ) de ) AARRCCHHITIETCECTUTURRE E NSI I NSS NSS 9971 - 9971 - 2494 2494 ( ( dp f dp ) f ) SSCCIEINENCCE E+ + TETCECHHNNOOLOLOGGY Y tirvn tlaA ot laA ot isrevinU yt isrevinU yt CCRROOSSSOOVEVRE R ceic fo o oohcS f l cS o oohcS i f l cS i ecne ecne DDOOCCTOTORRAAL L ecneicS retupmoC retupmoC ecneicS ecneicS DDISISSERERTATTAITOIONNS S DDOOCCTOTORRAAL L +hfbjia*GMFTSH9 +hfbjia*GMFTSH9 fi.otlaa.www . www a . a a l a t o l t . o fi . fi DDISISSERERTATTAITOIONNS S ot laA ytot laA isrevinU yt isrevinU 0202 0202 Aalto University publication series DOCTORAL DISSERTATIONS 9/2020 On Bilinear Techniques for Similarity Search and Boolean Matrix Multiplication Matti Karppa A doctoral dissertation completed for the degree of Doctor of Science (Technology) to be defended, with the permission of the Aalto University School of Science, at a public examination held at the lecture hall T2 of the school on 24 January 2020 at 12.
    [Show full text]
  • Download This PDF File
    T G¨ P 2012 C N Deadline: December 31, 2011 The Gödel Prize for outstanding papers in the area of theoretical computer sci- ence is sponsored jointly by the European Association for Theoretical Computer Science (EATCS) and the Association for Computing Machinery, Special Inter- est Group on Algorithms and Computation Theory (ACM-SIGACT). The award is presented annually, with the presentation taking place alternately at the Inter- national Colloquium on Automata, Languages, and Programming (ICALP) and the ACM Symposium on Theory of Computing (STOC). The 20th prize will be awarded at the 39th International Colloquium on Automata, Languages, and Pro- gramming to be held at the University of Warwick, UK, in July 2012. The Prize is named in honor of Kurt Gödel in recognition of his major contribu- tions to mathematical logic and of his interest, discovered in a letter he wrote to John von Neumann shortly before von Neumann’s death, in what has become the famous P versus NP question. The Prize includes an award of USD 5000. AWARD COMMITTEE: The winner of the Prize is selected by a committee of six members. The EATCS President and the SIGACT Chair each appoint three members to the committee, to serve staggered three-year terms. The committee is chaired alternately by representatives of EATCS and SIGACT. The 2012 Award Committee consists of Sanjeev Arora (Princeton University), Josep Díaz (Uni- versitat Politècnica de Catalunya), Giuseppe Italiano (Università a˘ di Roma Tor Vergata), Mogens Nielsen (University of Aarhus), Daniel Spielman (Yale Univer- sity), and Eli Upfal (Brown University). ELIGIBILITY: The rule for the 2011 Prize is given below and supersedes any di fferent interpretation of the parametric rule to be found on websites on both SIGACT and EATCS.
    [Show full text]
  • 40Th ACM Symposium on Theory of Computing (STOC 2008) Saturday
    40th ACM Symposium on Theory of Computing (STOC 2008) Saturday, May 17, 2008 7:00pm – 10:00pm Registration (Conference Centre) 8:00pm – 10:00pm Reception (Palm Court) Sunday, May 18, 2008 8:00am – 5:00pm Registration (Conference Centre) 8:10am – 8:35am Breakfast (Conference Centre) Session 1A Session 1B (Theatre) (Saanich Room) Chair: Venkat Guruswami Chair: David Shmoys (Cornell (University of Washington and University) Institute for Advanced Study) 8:35am - 8:55am Parallel Repetition in Projection The Complexity of Temporal Games and a Concentration Bound Constraint Satisfaction Problems Anup Rao Manuel Bodirsky, Jan Kara 9:00am - 9:20am SDP Gaps and UGC Hardness for An Effective Ergodic Theorem Multiway Cut, 0-Extension and and Some Applications Metric Labeling Satyadev Nandakumar Rajeskar Manokaran, Joseph (Seffi) Naor, Prasad Raghavendra, Roy Schwartz 9:25am – 9:45am Unique Games on Expanding Algorithms for Subset Selection Constraint Graphs are Easy in Linear Regression Sanjeev Arora, Subhash A. Khot, Abhimanyu Das, David Kempe Alexandra Kolla, David Steurer, Madhur Tulsiani, Nisheeth Vishnoi 9:45am - 10:10am Break Session 2 (Theatre) Chair: Joan Feigenbaum (Yale University) 10:10am - 11:10am Rethinking Internet Routing Invited talk by Jennifer Rexford (Princeton University) 11:10am - 11:20am Break Session 3A Session 3B (Theatre) (Saanich Room) Chair: Xiaotie Deng (City Chair: Anupam Gupta (Carnegie University of Hong Kong) Mellon University) The Pattern Matrix Method for 11:20am – 11:40am Interdomain Routing and Games Lower Bounds on Quantum Hagay Levin, Michael Schapira, Communication Aviv Zohar Alexander A. Sherstov 11:45am – 12:05pm Optimal approximation for the Classical Interaction Cannot Submodular Welfare Problem in Replace a Quantum Message the value oracle model Dmitry Gavinsky Jan Vondrak 12:10pm – 12:30pm Optimal Mechanism Design and Span-program-based quantum Money Burning algorithm for evaluating formulas Jason Hartline, Tim Ben W.
    [Show full text]
  • Session II – Unit I – Fundamentals
    Tutor:- Prof. Avadhoot S. Joshi Subject:- Design & Analysis of Algorithms Session II Unit I - Fundamentals • Algorithm Design – II • Algorithm as a Technology • What kinds of problems are solved by algorithms? • Evolution of Algorithms • Timeline of Algorithms Algorithm Design – II Algorithm Design (Cntd.) Figure briefly shows a sequence of steps one typically goes through in designing and analyzing an algorithm. Figure: Algorithm design and analysis process. Algorithm Design (Cntd.) Understand the problem: Before designing an algorithm is to understand completely the problem given. Read the problem’s description carefully and ask questions if you have any doubts about the problem, do a few small examples by hand, think about special cases, and ask questions again if needed. There are a few types of problems that arise in computing applications quite often. If the problem in question is one of them, you might be able to use a known algorithm for solving it. If you have to choose among several available algorithms, it helps to understand how such an algorithm works and to know its strengths and weaknesses. But often you will not find a readily available algorithm and will have to design your own. An input to an algorithm specifies an instance of the problem the algorithm solves. It is very important to specify exactly the set of instances the algorithm needs to handle. Algorithm Design (Cntd.) Ascertaining the Capabilities of the Computational Device: After understanding the problem you need to ascertain the capabilities of the computational device the algorithm is intended for. The vast majority of algorithms in use today are still destined to be programmed for a computer closely resembling the von Neumann machine—a computer architecture outlined by the prominent Hungarian-American mathematician John von Neumann (1903–1957), in collaboration with A.
    [Show full text]
  • Game Theory, Alive Anna R
    Game Theory, Alive Anna R. Karlin Yuval Peres 10.1090/mbk/101 Game Theory, Alive Game Theory, Alive Anna R. Karlin Yuval Peres AMERICAN MATHEMATICAL SOCIETY Providence, Rhode Island 2010 Mathematics Subject Classification. Primary 91A10, 91A12, 91A18, 91B12, 91A24, 91A43, 91A26, 91A28, 91A46, 91B26. For additional information and updates on this book, visit www.ams.org/bookpages/mbk-101 Library of Congress Cataloging-in-Publication Data Names: Karlin, Anna R. | Peres, Y. (Yuval) Title: Game theory, alive / Anna Karlin, Yuval Peres. Description: Providence, Rhode Island : American Mathematical Society, [2016] | Includes bibliographical references and index. Identifiers: LCCN 2016038151 | ISBN 9781470419820 (alk. paper) Subjects: LCSH: Game theory. | AMS: Game theory, economics, social and behavioral sciences – Game theory – Noncooperative games. msc | Game theory, economics, social and behavioral sciences – Game theory – Cooperative games. msc | Game theory, economics, social and behavioral sciences – Game theory – Games in extensive form. msc | Game theory, economics, social and behavioral sciences – Mathematical economics – Voting theory. msc | Game theory, economics, social and behavioral sciences – Game theory – Positional games (pursuit and evasion, etc.). msc | Game theory, economics, social and behavioral sciences – Game theory – Games involving graphs. msc | Game theory, economics, social and behavioral sciences – Game theory – Rationality, learning. msc | Game theory, economics, social and behavioral sciences – Game theory – Signaling,
    [Show full text]
  • Online Rank Aggregation
    JMLR: Workshop and Conference Proceedings 25:539{553, 2012 Asian Conference on Machine Learning Online Rank Aggregation Shota Yasutake [email protected] Panasonic System Networks Co., Ltd Kohei Hatano [email protected] Eiji Takimoto [email protected] Masayuki Takeda [email protected] Kyushu University Editor: Steven C.H. Hoi and Wray Buntine Abstract We consider an online learning framework where the task is to predict a permutation which represents a ranking of n fixed objects. At each trial, the learner incurs a loss defined as Kendall tau distance between the predicted permutation and the true permutation given by the adversary. This setting is quite natural in many situations such as information retrieval and recommendation tasks. We prove a lower bound of the cumulative loss and hardness results. Then, we propose an algorithm for this problem and prove its relative loss bound which shows our algorithm is close to optimal. Keywords: online learning, ranking, rank aggregation, permutation 1. Introduction The rank aggregation problem have gained much attention due to developments of infor- mation retrieval on the Internet, online shopping stores, recommendation systems and so on. The problem is, given m permutations of n fixed elements, to find a permutation that minimizes the sum of \distances" between itself and each given permutation. Here, each permutation represents a ranking over n elements. So, in other words, the ranking aggre- gation problem is to find an \average" ranking which reflects the characteristics of given rankings. In particular, the optimal ranking is called Kemeny optimal (Kemeny, 1959; Kemeny and Snell, 1962) when the distance is defined as Kendall tau distance (which we will define later).
    [Show full text]
  • The Boosting Apporach to Machine Learning
    Nonlinear Estimation and Classification, Springer, 2003. The Boosting Approach to Machine Learning An Overview Robert E. Schapire AT&T Labs Research Shannon Laboratory 180 Park Avenue, Room A203 Florham Park, NJ 07932 USA www.research.att.com/ schapire December 19, 2001 Abstract Boosting is a general method for improving the accuracy of any given learning algorithm. Focusing primarily on the AdaBoost algorithm, this chapter overviews some of the recent work on boosting including analyses of AdaBoost’s training error and generalization error; boosting’s connection to game theory and linear programming; the relationship between boosting and logistic regression; extensions of AdaBoost for multiclass classification problems; methods of incorporating human knowledge into boosting; and experimental and applied work using boosting. 1 Introduction Machine learning studies automatic techniques for learning to make accurate pre- dictions based on past observations. For example, suppose that we would like to build an email filter that can distinguish spam (junk) email from non-spam. The machine-learning approach to this problem would be the following: Start by gath- ering as many examples as posible of both spam and non-spam emails. Next, feed these examples, together with labels indicating if they are spam or not, to your favorite machine-learning algorithm which will automatically produce a classifi- cation or prediction rule. Given a new, unlabeled email, such a rule attempts to predict if it is spam or not. The goal, of course, is to generate a rule that makes the most accurate predictions possible on new test examples. 1 Building a highly accurate prediction rule is certainly a difficult task.
    [Show full text]
  • Doctor of Philosophy
    RICE UNIVERSITY By Randall Balestriero A THESIS SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE Doctor of Philosophy APPROVED, THESIS COMMITTEE Richard Baraniuk (Apr 28, 2021 06:57 ADT) Ankit B Patel (Apr 28, 2021 02:34 CDT) Ankit Patel Richard Baraniuk Behnam Azhang Behnam Azhang (Apr 26, 2021 18:34 CDT) Behnaam Aazhang Stephane Mallat Moshe Vardi Moshe Vardi (Apr 26, 2021 20:04 CDT) Albert Cohen (Apr 27, 2021 16:36 GMT+2) Moshe Vardi Albert Cohen HOUSTON, TEXAS April 2021 ABSTRACT Max-Affine Splines Insights Into Deep Learning by Randall Balestriero We build a rigorous bridge between deep networks (DNs) and approximation the- ory via spline functions and operators. Our key result is that a large class of DNs can be written as a composition of max-affine spline operators (MASOs), which provide a powerful portal through which to view and analyze their inner workings. For instance, conditioned on the spline partition region containing the input signal, the output of a MASO DN can be written as a simple affine transformation of the input. Studying the geometry of those regions allows to obtain novel insights into different regular- ization techniques, different layer configurations or different initialization schemes. Going further, this spline viewpoint allows to obtain precise geometric insights in various domains such as the characterization of the Deep Generative Networks's gen- erated manifold, the understanding of Deep Network pruning as a mean to simplify the DN input space partition or the relationship between different nonlinearities e.g. ReLU-Sigmoid Gated Linear Unit as simply corresponding to different MASO region membership inference algorithms.
    [Show full text]