DOCSLIB.ORG

Cynthia Rudin Research Interests Education

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Cynthia Rudin Research Interests Education Cynthia Rudin Professor of Computer Science, Electrical and Computer Engineering, and Statistical Science Duke University D342 LSRC Building, Box 90129, Durham, NC 27708 [email protected] https://users.cs.duke.edu/ cynthia/ Research Interests My work focuses on interpretable machine learning, which is important for responsible and trustworthy AI. This includes the design of algorithms for interpretable modeling, interpretable policy design, variable importance measures, causal infer- ence methods, ranking methods that assist with prioritization, uncertainty quantification, and methods that can incorporate domain-based constraints and other types of domain knowledge into machine learning. These techniques are applied to critical societal problems in criminology, healthcare, energy grid reliability, and other areas. The interpretable machine learning algorithms heavily rely on efficient discrete optimization techniques. Some accomplishments: My collaborators and I developed the first practical code for decision lists and decision trees that provably optimize • accuracy and sparsity. This work had an oral presentation at KDD 2017, a spotlight at NeurIPS 2019, and the latest work appeared in ICML 2020. Our work on optimal scoring systems (sparse linear models with integer coefficients) has been applied to several important • healthcare and criminal justice applications. For instance, my collaboration’s work on seizure prediction in ICU patients allowed for better allocation of ICU patient monitoring resources. This work won the 2019 INFORMS Innovative Applications in Analytics Award. It allowed doctors to monitor 2.8X more patients and saved over $6M at two major hospitals in FY2018. It helps to prevent severe brain damage in critically ill patients. I led a team in the first major effort to maintain an underground electrical distribution network using machine learning, in • joint work with Con Edison in NYC. This won the 2013 INFORMS Innovative Applications in Analytics Award. I solved a well-known (previously open) theoretical problem in machine learning as a PhD student, which is whether • AdaBoost maximizes the margin like SVM. Subsequent work solved a published COLT open problem, earning a prize. My collaborators and I developed code for detecting crime series in cities. This methodology (specifically, the Series • Finder algorithm) was adapted by the NYPD and their application (Patternizr) has been running live in NYC since 2016 for determining whether each new crime is related to past crimes. I have given invited and keynote talks at KDD (2014 and 2019), AISTATS, ECML-PKDD, ML in Healthcare, FAT-ML • (Fairness, Accountability, and Transparency), and at several other venues. I enjoy competing in data science competitions and coaching student teams. We have won awards in several competitions, • including the FICO Recognition Award for the first Explainable Machine Learning Competition in 2018, NTIRE Superresolution competition in 2018, PoeTix Literary Turing Competition in 2018, and we placed second in one of the ICML Exploration and Exploitation competitions. Education Ph.D.: Princeton University. Program in Applied and Computational Mathematics. Title: Boosting, Margins, and • Dynamics. Advisors: Ingrid Daubechies and Robert Schapire. BS/BA: University at Buffalo (SUNY), Honors Program. Outstanding Senior Award in the Arts and Sciences (one • awarded per year university-wide), separate outstanding senior awards from the Physics Department, Mathematics Department, and Music Department, BS Mathematical Physics, BA Music Theory, Minor in Computer Science, Summa Cum Laude, 1999 Cynthia Rudin 2 Employment History Duke University, Computer Science Department (50%), Electrical and Computer Engineering Department (50%), Sec- • ondary appointment in Statistical Science, Secondary appointment in Mathematics. Associate Professor 2016-2019, Professor 2019-present Massachusetts Institute of Technology, MIT Computer Science and Artificial Intelligence Laboratory and Sloan School • of Management, Associate Professor of Statistics 2013-2016, Assistant Professor of Statistics 2009-2013. Columbia University, Center for Computational Learning Systems, Associate Research Scientist, 2007-2009. • NSF Postdoctoral Research Fellow, New York University, 2004-2007 • Princeton University, Graduate Research Assistant, Program in Applied and Computational Mathematics, September • 1999 - June 2004, Postdoctoral Research Fellow, summer 2004 Peer-Reviewed Publications Papers associated with major awards (winner and finalist) 1. Gah-Yi Ban and Cynthia Rudin. MSOM Best OM paper in OR Award, INFORMS. (Awarded to the best paper in Oper- ations Research within the last 3 years, awarded by Manufacturing and Service Operations Management Society of INFORMS.) The Big Data Newsvendor: Practical Insights from Machine Learning, Operations Research, Vol. 67, No. 1, pages 90-108, 2019. 2. Aaron F. Struck, Berk Ustun, Andres Rodriguez Ruiz, Jong Woo Lee, Suzette LaRoche, Lawrence J. Hirsch, Emily J Gilmore, Jan Vlachy, Hiba Arif Haider, Cynthia Rudin, M Brandon Westover. 2019 INFORMS Innovative Applica- tions in Analytics Award (shared with paper below). Association of an Electroencephalography-Based Risk Score With Seizure Probability in Hospitalized Patients, JAMA Neurology, 74 (12), 1419-1424, 2017. 3. Berk Ustun and Cynthia Rudin. 2019 INFORMS Innovative Applications in Analytics Award, also 2017 INFORMS Com- puting Society Student Paper Prize. Learning Optimized Risk Scores. Journal of Machine Learning Research, 2019. Shorter version Learning Optimized Risk Scores from Large-Scale Datasets. Knowledge Discovery in Databases (KDD), 2017. 4. Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, Tong Wang. A Holistic Approach to Interpretability in Financial Lending: Models, Visualizations, and Summary-Explanations. Decision Support Systems, Accepted, 2021. Preliminary work: Chaofan Chen, Kangcheng Lin, Cynthia Rudin, Yaron Shaposhnik, Sijia Wang, Tong Wang. Winner of the FICO • Recognition Award for the Explainable Machine Learning Challenge, 2018. An Interpretable Model with Globally Consistent Explanations for Credit Risk. NIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Services: the Impact of Fairness, Explainability, Accuracy, and Privacy, 2018. 5. Cynthia Rudin and Berk Ustun. Finalist for 2017 Daniel H. Wagner Prize for Excellence in Operations Research, Institute for Operations Research and Management Science (INFORMS). Optimized Scoring Systems: Towards Trust in Machine Learning for Healthcare and Criminal Justice. INFORMS Journal on Applied Analytics, Special Issue: 2017 Daniel H. Wagner Prize for Excellence in Operations Research Practice, 48(5), pages 399–486, September- October, 2018. 6. William Souillard-Mandar, Randall Davis, Cynthia Rudin, Rhoda Au, David J. Libon, Rodney Swenson, Catherine C. Price, Melissa Lamar, Dana L. Penney. 2016 INFORMS Innovative Applications in Analytics Award (shared with paper below). Learning Classification Models of Cognitive Conditions from Subtle Behaviors in the Digital Clock Drawing Test. Machine Learning, volume 102, number 3, 2016. 7. Berk Ustun and Cynthia Rudin. 2016 INFORMS Innovative Applications in Analytics Award and also Runner up, Invenia Labs SEE Award 2018 - Supporting Machine Learning Research with a Positive Impact on Social, Economic, or Environmental (SEE) Challenges. Supersparse Linear Integer Models for Optimized Medical Scoring Systems. Machine Learning, volume 102, number 3, 2016. Cynthia Rudin 3 8. Cynthia Rudin, S¸eyda Ertekin, Rebecca Passonneau, Axinia Radeva, Ashish Tomar, Boyi Xie, Stanley Lewis, Mark Riddle, Debbie Pangsrivinij, Tyler McCormick. 2013 INFORMS Innovative Applications in Analytics Award Analytics for Power Grid Distribution Reliability in New York City. INFORMS Journal on Applied Analytics, volume 44, issue 4, pages 364-383, 2014. 9. Indraneel Mukherjee, Cynthia Rudin, and Robert E. Schapire. This paper answered an open question published in COLT 2010. The Rate of Convergence of AdaBoost, Journal of Machine Learning Research, volume 14, pages 2315-2347, August 2013. Preliminary work: Indraneel Mukherjee, Cynthia Rudin, and Robert E. Schapire. The Rate of Convergence of AdaBoost, Proceedings • of the 24th Annual Conference on Learning Theory (COLT), 2011. Peer-Reviewed Publications (not including those above) 2021 10. Tong Wang and Cynthia Rudin. Subgroup Identification for Enhanced Treatment Effect with Decision Rules, IN- FORMS Journal on Computing, Accepted, 2021. 11. Yingfan Wang, Haiyang Huang, Cynthia Rudin, and Yaron Shaposhnik. Understanding How Dimension Reduction Tools Work: An Empirical Approach to Deciphering t-SNE, UMAP, TriMAP, and PaCMAP for Data Visual- ization, Journal of Machine Learning Research, accepted with minor revision, 2021. 12. Tianyu Wang, Marco Morucci, M. Usaid Awan, Yameng Liu, Sudeepa Roy, Cynthia Rudin, Alexander Volfovsky. FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference. Journal of Machine Learning Re- search, 2021. 13. Stefano Traca`, Cynthia Rudin, Weiyu Yan. (Best paper award, INFORMS 2016 Data Mining & Decision Analytics Workshop, also Finalist for 2015 IBM Service Science Best Paper Award.) Regulating Greed over Time in Multi- Armed Bandits. Journal of Machine Learning Research, 2021. 14. Jianyou Wang, Xiaoxuan Zhang, Yuren Zhou, Christopher Suh, Cynthia Rudin. There once was a really bad poet, it was automated but you didn’t know it. Transactions of the Association for
Load more

Recommended publications
  • Relational Machine Learning Algorithms
    Relational Machine Learning Algorithms by Alireza Samadianzakaria Bachelor of Science, Sharif University of Technology, 2016 Submitted to the Graduate Faculty of the Department of Computer Science in partial fulfillment of the requirements for the degree of Doctor of Philosophy University of Pittsburgh 2021 UNIVERSITY OF PITTSBURGH DEPARTMENT OF COMPUTER SCIENCE This dissertation was presented by Alireza Samadianzakaria It was defended on July 7, 2021 and approved by Dr. Kirk Pruhs, Department of Computer Science, University of Pittsburgh Dr. Panos Chrysanthis, Department of Computer Science, University of Pittsburgh Dr. Adriana Kovashka, Department of Computer Science, University of Pittsburgh Dr. Benjamin Moseley, Tepper School of Business, Carnegie Mellon University ii Copyright c by Alireza Samadianzakaria 2021 iii Relational Machine Learning Algorithms Alireza Samadianzakaria, PhD University of Pittsburgh, 2021 The majority of learning tasks faced by data scientists involve relational data, yet most standard algorithms for standard learning problems are not designed to accept relational data as input. The standard practice to address this issue is to join the relational data to create the type of geometric input that standard learning algorithms expect. Unfortunately, this standard practice has exponential worst-case time and space complexity. This leads us to consider what we call the Relational Learning Question: \Which standard learning algorithms can be efficiently implemented on relational data, and for those that cannot, is there an alternative algorithm that can be efficiently implemented on relational data and that has similar performance guarantees to the standard algorithm?" In this dissertation, we address the relational learning question for the well-known prob- lems of support vector machine (SVM), logistic regression, and k-means clustering.
    [Show full text]
  • Computer Science and Decision Theory Fred S. Roberts1 Abstract 1
    Computer Science and Decision Theory Fred S. Roberts1 DIMACS Center, Rutgers University, Piscataway, NJ 08854 USA [email protected] Abstract This paper reviews applications in computer science that decision theorists have addressed for years, discusses the requirements posed by these applications that place great strain on decision theory/social science methods, and explores applications in the social and decision sciences of newer decision-theoretic methods developed with computer science applications in mind. The paper deals with the relation between computer science and decision-theoretic methods of consensus, with the relation between computer science and game theory and decisions, and with \algorithmic decision theory." 1 Introduction Many applications in computer science involve issues and problems that decision theorists have addressed for years, issues of preference, utility, conflict and cooperation, allocation, incentives, consensus, social choice, and measurement. A similar phenomenon is apparent more generally at the interface between computer sci- ence and the social sciences. We have begun to see the use of methods developed by decision theorists/social scientists in a variety of computer science applications. The requirements posed by these computer science applications place great strain on the decision theory/social science methods because of the sheer size of the problems addressed, new contexts in which computational power of agents becomes an issue, limitations on information possessed by players, and the sequential nature of repeated applications. Hence, there is a great need to develop a new generation of methods to satisfy these computer science requirements. In turn, these new methods will provide powerful new tools for social scientists in general and decision theorists in particular.
    [Show full text]
  • Research Notices
    AMERICAN MATHEMATICAL SOCIETY Research in Collegiate Mathematics Education. V Annie Selden, Tennessee Technological University, Cookeville, Ed Dubinsky, Kent State University, OH, Guershon Hare I, University of California San Diego, La jolla, and Fernando Hitt, C/NVESTAV, Mexico, Editors This volume presents state-of-the-art research on understanding, teaching, and learning mathematics at the post-secondary level. The articles are peer-reviewed for two major features: (I) advancing our understanding of collegiate mathematics education, and (2) readability by a wide audience of practicing mathematicians interested in issues affecting their students. This is not a collection of scholarly arcana, but a compilation of useful and informative research regarding how students think about and learn mathematics. This series is published in cooperation with the Mathematical Association of America. CBMS Issues in Mathematics Education, Volume 12; 2003; 206 pages; Softcover; ISBN 0-8218-3302-2; List $49;AII individuals $39; Order code CBMATH/12N044 MATHEMATICS EDUCATION Also of interest .. RESEARCH: AGul<lelbrthe Mathematics Education Research: Hothomatldan- A Guide for the Research Mathematician --lllll'tj.M...,.a.,-- Curtis McKnight, Andy Magid, and -- Teri J. Murphy, University of Oklahoma, Norman, and Michelynn McKnight, Norman, OK 2000; I 06 pages; Softcover; ISBN 0-8218-20 16-8; List $20;AII AMS members $16; Order code MERN044 Teaching Mathematics in Colleges and Universities: Case Studies for Today's Classroom Graduate Student Edition Faculty
    [Show full text]
  • An Efficient Boosting Algorithm for Combining Preferences Raj
    An Efficient Boosting Algorithm for Combining Preferences by Raj Dharmarajan Iyer Jr. Submitted to the Department of Electrical Engineering and Computer Science in partial fulfillment of the requirements for the degree of Master of Science at the MASSACHUSETTS INSTITUTE OF TECHNOLOGY September 1999 © Massachusetts Institute of Technology 1999. All rights reserved. A uth or .............. ..... ..... .................................. Department of E'ectrical ngineering and Computer Science August 24, 1999 C ertified by .. ................. ...... .. .............. David R. Karger Associate Professor Thesis Supervisor Accepted by............... ........ Arthur C. Smith Chairman, Departmental Committe Graduate Students MACHU OF TEC Lo NOV LIBRARIES An Efficient Boosting Algorithm for Combining Preferences by Raj Dharmarajan Iyer Jr. Submitted to the Department of Electrical Engineering and Computer Science on August 24, 1999, in partial fulfillment of the requirements for the degree of Master of Science Abstract The problem of combining preferences arises in several applications, such as combining the results of different search engines. This work describes an efficient algorithm for combining multiple preferences. We first give a formal framework for the problem. We then describe and analyze a new boosting algorithm for combining preferences called RankBoost. We also describe an efficient implementation of the algorithm for certain natural cases. We discuss two experiments we carried out to assess the performance of RankBoost. In the first experi- ment, we used the algorithm to combine different WWW search strategies, each of which is a query expansion for a given domain. For this task, we compare the performance of Rank- Boost to the individual search strategies. The second experiment is a collaborative-filtering task for making movie recommendations.
    [Show full text]
  • An Introduction to Johnson–Lindenstrauss Transforms
    An Introduction to Johnson–Lindenstrauss Transforms Casper Benjamin Freksen∗ 2nd March 2021 Abstract Johnson–Lindenstrauss Transforms are powerful tools for reducing the dimensionality of data while preserving key characteristics of that data, and they have found use in many fields from machine learning to differential privacy and more. This note explains whatthey are; it gives an overview of their use and their development since they were introduced in the 1980s; and it provides many references should the reader wish to explore these topics more deeply. The text was previously a main part of the introduction of my PhD thesis [Fre20], but it has been adapted to be self contained and serve as a (hopefully good) starting point for readers interested in the topic. 1 The Why, What, and How 1.1 The Problem Consider the following scenario: We have some data that we wish to process but the data is too large, e.g. processing the data takes too much time, or storing the data takes too much space. A solution would be to compress the data such that the valuable parts of the data are kept and the other parts discarded. Of course, what is considered valuable is defined by the data processing we wish to apply to our data. To make our scenario more concrete let us say that our data consists of vectors in a high dimensional Euclidean space, R3, and we wish to find a transform to embed these vectors into a lower dimensional space, R<, where < 3, so that we arXiv:2103.00564v1 [cs.DS] 28 Feb 2021 ≪ can apply our data processing in this lower dimensional space and still get meaningful results.
    [Show full text]
  • Download This PDF File
    T G¨ P 2012 C N Deadline: December 31, 2011 The Gödel Prize for outstanding papers in the area of theoretical computer sci- ence is sponsored jointly by the European Association for Theoretical Computer Science (EATCS) and the Association for Computing Machinery, Special Inter- est Group on Algorithms and Computation Theory (ACM-SIGACT). The award is presented annually, with the presentation taking place alternately at the Inter- national Colloquium on Automata, Languages, and Programming (ICALP) and the ACM Symposium on Theory of Computing (STOC). The 20th prize will be awarded at the 39th International Colloquium on Automata, Languages, and Pro- gramming to be held at the University of Warwick, UK, in July 2012. The Prize is named in honor of Kurt Gödel in recognition of his major contribu- tions to mathematical logic and of his interest, discovered in a letter he wrote to John von Neumann shortly before von Neumann’s death, in what has become the famous P versus NP question. The Prize includes an award of USD 5000. AWARD COMMITTEE: The winner of the Prize is selected by a committee of six members. The EATCS President and the SIGACT Chair each appoint three members to the committee, to serve staggered three-year terms. The committee is chaired alternately by representatives of EATCS and SIGACT. The 2012 Award Committee consists of Sanjeev Arora (Princeton University), Josep Díaz (Uni- versitat Politècnica de Catalunya), Giuseppe Italiano (Università a˘ di Roma Tor Vergata), Mogens Nielsen (University of Aarhus), Daniel Spielman (Yale Univer- sity), and Eli Upfal (Brown University). ELIGIBILITY: The rule for the 2011 Prize is given below and supersedes any di fferent interpretation of the parametric rule to be found on websites on both SIGACT and EATCS.
    [Show full text]
  • Hardness of Learning Halfspaces with Noise∗
    Hardness of Learning Halfspaces with Noise∗ Venkatesan Guruswamiy Prasad Raghavendraz Department of Computer Science and Engineering University of Washington Seattle, WA 98195 Abstract Learning an unknown halfspace (also called a perceptron) from labeled examples is one of the classic problems in machine learning. In the noise-free case, when a halfspace consistent with all the training examples exists, the problem can be solved in polynomial time using linear programming. However, under the promise that a halfspace consistent with a fraction (1 − ") of the examples exists (for some small constant " > 0), it was not known how to efficiently find a halfspace that is correct on even 51% of the examples. Nor was a hardness result that ruled out getting agreement on more than 99:9% of the examples known. In this work, we close this gap in our understanding, and prove that even a tiny amount of worst-case noise makes the problem of learning halfspaces intractable in a strong sense. Specifically, for arbitrary "; δ > 0, we prove that given a set of examples-label pairs from the hypercube a fraction (1 − ") of which can be explained by a halfspace, it is NP-hard to find a halfspace that correctly labels a fraction (1=2 + δ) of the examples. The hardness result is tight since it is trivial to get agreement on 1=2 the examples. In learning theory parlance, we prove that weak proper agnostic learning of halfspaces is hard. This settles a question that was raised by Blum et al. in their work on learning halfspaces in the presence of random classification noise [10], and in some more recent works as well.
    [Show full text]
  • Optimal Bounds for Estimating Entropy with PMF Queries
    Optimal Bounds for Estimating Entropy with PMF Queries Cafer Caferov1, Barı¸sKaya1, Ryan O'Donnell2?, and A. C. Cem Say1 1 Bo˘gazi¸ciUniversity Computer Engineering Department fcafer.caferov,baris.kaya,[email protected] 2 Department of Computer Science, Carnegie Mellon University [email protected] Abstract. Let p be an unknown probability distribution on [n] := f1; 2; : : : ng that we can access via two kinds of queries: A SAMP query takes no input and returns x 2 [n] with probability p[x]; a PMF query takes as input x 2 [n] and returns the value p[x]. We consider the task of estimating the entropy of p to within ±∆ (with high probability). For the usual Shannon entropy H(p), we show that Ω(log2 n=∆2) queries are necessary, matching a recent upper bound of Canonne and Rubinfeld. For 1−1/α the R´enyi entropy Hα(p), where α > 1, we show that Θ n queries are necessary and sufficient. This complements recent work of Acharya et al. in the SAMP-only model that showed O(n1−1/α) queries suffice when α is an integer, but Ωe (n) queries are necessary when α is a non- integer. All of our lower bounds also easily extend to the model where P CDF queries (given x, return y≤x p[y]) are allowed. 1 Introduction The field of statistics is to a large extent concerned with questions of the following sort: How many samples from an unknown probability distribution p are needed in order to accurately estimate various properties of the distribution? These sorts of questions have also been studied more recently within the theoretical computer science framework of property testing.
    [Show full text]
  • Mitigating Bias in Adaptive Data Gathering Via Differential Privacy
    Mitigating Bias in Adaptive Data Gathering via Differential Privacy Seth Neel∗ Aaron Rothy June 7, 2018 Abstract Data that is gathered adaptively | via bandit algorithms, for example | exhibits bias. This is true both when gathering simple numeric valued data | the empirical means kept track of by stochastic bandit algorithms are biased downwards | and when gathering more complicated data | running hypothesis tests on complex data gathered via contextual bandit algorithms leads to false discovery. In this paper, we show that this problem is mitigated if the data collection procedure is differentially private. This lets us both bound the bias of simple numeric valued quantities (like the empirical means of stochastic bandit algorithms), and correct the p-values of hypothesis tests run on the adaptively gathered data. Moreover, there exist differentially private bandit algorithms with near optimal regret bounds: we apply existing theorems in the simple stochastic case, and give a new analysis for linear contextual bandits. We complement our theoretical results with experiments validating our theory. arXiv:1806.02329v1 [cs.LG] 6 Jun 2018 ∗Department of Statistics, The Wharton School, University of Pennsylvania. [email protected]. Supported in part by a 2017 NSF Graduate Research Fellowship. yDepartment of Computer and Information Sciences, University of Pennsylvania. [email protected]. Sup- ported in part by grants from the DARPA Brandeis project, the Sloan Foundation, and NSF grants CNS-1513694 and CNS-1253345. 1 Introduction Many modern data sets consist of data that is gathered adaptively: the choice of whether to collect more data points of a given type depends on the data already collected.
    [Show full text]
  • Arxiv:2008.13374V2
    Active Local Learning Arturs Backurs [email protected] Toyota Technological Institute at Chicago Avrim Blum [email protected] Toyota Technological Institute at Chicago Neha Gupta [email protected] Stanford University Abstract In this work we consider active local learning: given a query point x, and active access to an unlabeled training set S, output the prediction h(x) of a near-optimal h ∈ H using significantly fewer labels than would be needed to actually learn h fully. In particular, the number of label queries should be independent of the complexity of H, and the function h should be well-defined, independent of x. This immediately also implies an algorithm for distance estimation: estimating the value opt(H) from many fewer labels than needed to actually learn a near-optimal h ∈ H, by running local learning on a few random query points and computing the average error. For the hypothesis class consisting of functions supported on the interval [0, 1] with Lipschitz constant bounded by L, we present an algorithm that makes O((1/ǫ6) log(1/ǫ)) label queries from an unlabeled pool of O((L/ǫ4) log(1/ǫ)) samples. It estimates the distance to the best hypothesis in the class to an additive error of ǫ for an arbitrary underlying distribution. We further generalize our algorithm to more than one dimensions. We emphasize that the number of labels used is independent of the complexity of the hypothesis class which is linear in L in the one-dimensional case. Furthermore, we give an algorithm to locally estimate the values of a near-optimal function at a few query points of interest with number of labels independent of L.
    [Show full text]
  • The Boosting Apporach to Machine Learning
    Nonlinear Estimation and Classification, Springer, 2003. The Boosting Approach to Machine Learning An Overview Robert E. Schapire AT&T Labs Research Shannon Laboratory 180 Park Avenue, Room A203 Florham Park, NJ 07932 USA www.research.att.com/ schapire December 19, 2001 Abstract Boosting is a general method for improving the accuracy of any given learning algorithm. Focusing primarily on the AdaBoost algorithm, this chapter overviews some of the recent work on boosting including analyses of AdaBoost’s training error and generalization error; boosting’s connection to game theory and linear programming; the relationship between boosting and logistic regression; extensions of AdaBoost for multiclass classification problems; methods of incorporating human knowledge into boosting; and experimental and applied work using boosting. 1 Introduction Machine learning studies automatic techniques for learning to make accurate pre- dictions based on past observations. For example, suppose that we would like to build an email filter that can distinguish spam (junk) email from non-spam. The machine-learning approach to this problem would be the following: Start by gath- ering as many examples as posible of both spam and non-spam emails. Next, feed these examples, together with labels indicating if they are spam or not, to your favorite machine-learning algorithm which will automatically produce a classifi- cation or prediction rule. Given a new, unlabeled email, such a rule attempts to predict if it is spam or not. The goal, of course, is to generate a rule that makes the most accurate predictions possible on new test examples. 1 Building a highly accurate prediction rule is certainly a difficult task.
    [Show full text]
  • Artists Routinely Provide an Unique
    rsity Comp ve ut ni er U S c n ie w n o c r e B 2 5 4 t 0 h 9- AN 7 NIVERSARY conduit! Volume 13, Number 1 Department of Computer Science Summer, 2004 25th Anniversary issue Brown University technologies, such as virtual reality (VR) or vol- ume rendering. It is difficult for artists to get in- ARTISTIC COLLABORATION volved in design in these visual spaces since, with rare exceptions, one needs to know how to pro- IN DESIGNING gram in order to create within them. Unfortunate- VR VISUALIZATIONS ly, these are also the types of technologies that offer great potential for visualizing many of to- day’s complex datasets [4]. Additionally, they are probably the technologies in which we can most benefit from artistic insight, since guidelines for good visual depiction are far less developed in un- conventional visual spaces, such as virtual reality, than in more traditional 2D media. We begin by describing one of our recent major collaborative efforts, a class on designing virtual Daniel Keefe David Karelitz Eileen Vote David Laidlaw reality scientific visualizations that was co-taught with professors and students from Brown’s com- This article describes some of the lessons we have puter science department and from the Rhode Is- learned from our collaborations with artists on land School of Design (RISD)’s illustration visualization problems. Over the past several department. Many of the experiences and conclu- years, we have worked closely with artists to de- sions relayed here are the results of this class. We velop, refine, and critique visualizations ranging then discuss three important themes that we have from archaeological dig data from the Great Temple of Petra site in Jordan to the fluid dynamics and wing bone shape defor- artists routinely provide an mations that begin to explain how bats, the only flying mammals, stay aloft.
    [Show full text]