End-To-End Large Scale Machine Learning with Keystoneml by Evan

Total Page:16

File Type:pdf, Size:1020Kb

End-To-End Large Scale Machine Learning with Keystoneml by Evan End-to-End Large Scale Machine Learning with KeystoneML by Evan Randall Sparks A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Michael J. Franklin, Co-chair Professor Benjamin Recht, Co-chair Professor Ion Stoica Professor Joshua S. Bloom Fall 2016 End-to-End Large Scale Machine Learning with KeystoneML Copyright 2016 by Evan Randall Sparks 1 Abstract End-to-End Large Scale Machine Learning with KeystoneML by Evan Randall Sparks Doctor of Philosophy in Computer Science University of California, Berkeley Professor Michael J. Franklin, Co-chair Professor Benjamin Recht, Co-chair The rise of data center computing and Internet-connected devices has led to an unparal- leled explosion in the volumes of data collected across a multitude of industries and academic disciplines. This data serves as fuel for statistical machine learning techniques that in turn enable some of today's most advanced applications including those powered by image clas- sification, speech recognition, and natural language understanding, which we broadly term machine learning applications. Unfortunately, until recently the tools and techniques used to leverage recent advances in machine learning at the scales demanded by modern datasets, and thus develop these applications, have been available only to experts in fields such as distributed computing, statistics, and optimization. I describe my efforts to render these tools accessible to a broader audience of applica- tion developers, and further demonstrate that by taking a holistic approach and capturing end-to-end high level specifications of machine learning applications the systems I present here can make novel, high impact optimizations to decrease resource consumption while simultaneously increasing throughput. These improvements are designed to decrease ML application development time, increase quality, and increase machine learning application developer productivity. I demonstrate the viability of these optimizations via experiments on a number of real-world applications in domains such as collaborative filtering, computer vision, and natural language processing. Many of the ideas presented in this thesis have already had practical impact as embodied in the open source software packages KeystoneML and Apache Spark MLlib. i For my family. ii Contents Contents ii List of Figures vi List of Tables ix 1 Introduction 1 1.1 Datacenter Explosion . 2 1.2 The Promise of Machine Learning . 3 1.3 Machine Learning Pipelines . 3 1.4 The Challenges of Large Scale ML . 4 1.5 Decisions, Decisions . 6 1.6 Thesis Overview and Contributions . 7 1.7 Organization . 8 2 Background 10 2.1 Machine Learning Application Development . 10 2.1.1 Data Collection . 11 2.1.2 Data Preprocessing and Featurization . 12 2.1.3 Machine Learning Algorithms . 13 2.1.4 Deployment . 15 2.1.5 Overfitting and Hyperparameter Tuning . 15 2.1.6 Scaling Up Machine Learning . 17 2.2 Related Work . 19 2.2.1 Learning Systems . 19 2.2.2 Large Scale Learning . 19 2.2.3 Hyperparameter Tuning . 20 2.2.4 Declarative Programming and Database Optimization . 22 2.2.5 Non-Goals: Generative Models, Tiny Data, and Deep Learning . 22 2.2.6 Other Related Work . 23 3 MLI: An API for Distributed Machine Learning 25 iii 3.1 Introduction . 25 3.2 The MLI Interface . 27 3.2.1 MLTable . 28 3.2.2 LocalMatrix . 28 3.2.3 Optimization, Models, and Algorithms . 29 3.3 Examples . 30 3.3.1 Binary Classification: Logistic Regression . 33 3.3.2 Collaborative Filtering: Alternating Least Squares . 35 3.3.3 Configuration Considerations . 39 3.4 Related work . 40 3.5 Conclusion . 41 4 TuPAQ: Automating Model Search for Large Scale Machine Learning 42 4.1 Introduction . 42 4.2 Model Search and TUPAQ............................ 44 4.2.1 Defining Model Search . 44 4.2.2 Problem Setting . 45 4.2.3 Connections to Query Optimization . 46 4.2.4 Baseline Model Search . 47 4.2.5 TuPAQ Model Search . 47 4.3 TUPAQ Design Choices . 51 4.3.1 Cost-based Execution Strategy Selection . 51 4.3.2 Advanced Hyperparameter Tuning . 53 4.3.3 Bandit Resource Allocation . 54 4.3.4 Batching . 55 4.4 Design Space Evaluation . 58 4.4.1 Execution Strategy Selection . 58 4.4.2 Hyperparameter Tuning . 60 4.4.3 Bandit Resource Allocation . 62 4.4.4 Batching . 63 4.5 Putting It All Together . 64 4.5.1 Platform Configuration . 65 4.5.2 Experimental Setup and Datasets . 65 4.5.3 Optimization Effects . 66 4.5.4 Large Scale Speech and Vision . 67 4.6 Related Work . 68 4.7 Future Work and Conclusions . 69 5 KeystoneML: Optimizing Pipelines for Large-Scale Advanced Analytics 71 5.1 Introduction . 71 5.2 Pipeline Construction and Core APIs . 74 5.2.1 Logical ML Operators . 74 iv 5.2.2 Pipeline Construction . 75 5.2.3 Pipeline Execution . 76 5.3 Operator-Level Optimization . 76 5.4 Whole-Pipeline Optimization . 81 5.4.1 Execution Subsampling . 81 5.4.2 Common Sub-expression Elimination . 81 5.4.3 Automatic Materialization . 81 5.5 Evaluation . 84 5.5.1 End-to-End ML Applications . 85 5.5.2 KeystoneML vs. Other Systems . 86 5.5.3 Optimization Levels . 89 5.5.4 Automatic Materialization Strategies . 90 5.5.5 Scalability . 91 5.6 Related Work . 92 5.7 Future Work and Conclusion . 93 6 Piperplanned: Resource Aware Pipeline Hyperparameter Tuning 95 6.1 Introduction . 95 6.2 Hyperparameter Optimization . 98 6.2.1 Hyperparameter Search . 98 6.2.2 Pipelines . 99 6.2.3 Pipeline Tuning . 99 6.3 Implementation . 100 6.3.1 Search Space Description . 100 6.3.2 Operator Graphs . 102 6.3.3 Execution Planner . 102 6.4 Optimal Reuse in Pipeline Execution . 103 6.4.1 Maximum Achievable Reuse . 103 6.4.2 Memory Constraints . 104 6.4.3 Optimal Dataflow Reuse with Constrained Memory . 106 6.5 Evaluation . 108 6.5.1 Experimental Setup . 108 6.5.2 Limitations on Solving the ILP . 108 6.5.3 Evaluating Cache Strategies . 109 6.5.4 Results on Real Hyperparameter Configurations . 111 6.6 Related Work . 113 6.7 Future Work and Conclusions . 114 7 Future Work and Conclusions 115 7.1 Contributions . 115 7.2 Limitations . 116 7.3 Future Challenges . 117 v 7.4 Final Remarks . 118 Bibliography 119 vi List of Figures 1.1 Cost of storage over time [109]. 2 1.2 A \simple" image classification pipeline. 4 1.3 A \simple" image classification pipeline annotated by ease of scalability. 5 1.4 A \simple" image classification pipeline annotated with hyperparameters. 7 2.1 A high level view of the ML application development cycle. 11 3.1 Landscape of existing development platforms for ML. 26 3.2 MLTable API.
Recommended publications
  • The Shogun Machine Learning Toolbox
    The Shogun Machine Learning Toolbox Heiko Strathmann, Gatsby Unit, UCL London Open Machine Learning Workshop, MSR, NY August 22, 2014 A bit about Shogun I Open-Source tools for ML problems I Started 1999 by SÖren Sonnenburg & GUNnar Rätsch, made public in 2004 I Currently 8 core-developers + 20 regular contributors I Purely open-source community driven I In Google Summer of Code since 2010 (29 projects!) Ohloh - Summary Ohloh - Code Supervised Learning Given: x y n , want: y ∗ x ∗ I f( i ; i )gi=1 j I Classication: y discrete I Support Vector Machine I Gaussian Processes I Logistic Regression I Decision Trees I Nearest Neighbours I Naive Bayes I Regression: y continuous I Gaussian Processes I Support Vector Regression I (Kernel) Ridge Regression I (Group) LASSO Unsupervised Learning Given: x n , want notion of p x I f i gi=1 ( ) I Clustering: I K-Means I (Gaussian) Mixture Models I Hierarchical clustering I Latent Models I (K) PCA I Latent Discriminant Analysis I Independent Component Analysis I Dimension reduction I (K) Locally Linear Embeddings I Many more... And many more I Multiple Kernel Learning I Structured Output I Metric Learning I Variational Inference I Kernel hypothesis testing I Deep Learning (whooo!) I ... I Bindings to: LibLinear, VowpalWabbit, etc.. http://www.shogun-toolbox.org/page/documentation/ notebook Some Large-Scale Applications I Splice Site prediction: 50m examples of 200m dimensions I Face recognition: 20k examples of 750k dimensions ML in Practice I Modular data represetation I Dense, Sparse, Strings, Streams, ... I Multiple types: 8-128 bit word size I Preprocessing tools I Evaluation I Cross-Validation I Accuracy, ROC, MSE, ..
    [Show full text]
  • Python Libraries, Development Frameworks and Algorithms for Machine Learning Applications
    Published by : International Journal of Engineering Research & Technology (IJERT) http://www.ijert.org ISSN: 2278-0181 Vol. 7 Issue 04, April-2018 Python Libraries, Development Frameworks and Algorithms for Machine Learning Applications Dr. V. Hanuman Kumar, Sr. Assoc. Prof., NHCE, Bangalore-560 103 Abstract- Nowadays Machine Learning (ML) used in all sorts numerical values and is of the form y=A0+A1*x, for input of fields like Healthcare, Retail, Travel, Finance, Social Media, value x and A0, A1 are the coefficients used in the etc., ML system is used to learn from input data to construct a representation with the data that we have. The classification suitable model by continuously estimating, optimizing and model helps to identify the sentiment of a particular post. tuning parameters of the model. To attain the stated, For instance a user review can be classified as positive or python programming language is one of the most flexible languages and it does contain special libraries for negative based on the words used in the comments. Emails ML applications namely SciPy, NumPy, etc., which great for can be classified as spam or not, based on these models. The linear algebra and getting to know kernel methods of machine clustering model helps when we are trying to find similar learning. The python programming language is great to use objects. For instance If you are interested in read articles when working with ML algorithms and has easy syntax about chess or basketball, etc.., this model will search for the relatively. When taking the deep-dive into ML, choosing a document with certain high priority words and suggest framework can be daunting.
    [Show full text]
  • Naiad: a Timely Dataflow System
    Naiad: A Timely Dataflow System Derek G. Murray Frank McSherry Rebecca Isaacs Michael Isard Paul Barham Mart´ın Abadi Microsoft Research Silicon Valley {derekmur,mcsherry,risaacs,misard,pbar,abadi}@microsoft.com Abstract User queries Low-latency query are received responses are delivered Naiad is a distributed system for executing data parallel, cyclic dataflow programs. It offers the high throughput Queries are of batch processors, the low latency of stream proces- joined with sors, and the ability to perform iterative and incremental processed data computations. Although existing systems offer some of Complex processing these features, applications that require all three have re- incrementally re- lied on multiple platforms, at the expense of efficiency, Updates to executes to reflect maintainability, and simplicity. Naiad resolves the com- data arrive changed data plexities of combining these features in one framework. A new computational model, timely dataflow, under- Figure 1: A Naiad application that supports real- lies Naiad and captures opportunities for parallelism time queries on continually updated data. The across a wide class of algorithms. This model enriches dashed rectangle represents iterative processing that dataflow computation with timestamps that represent incrementally updates as new data arrive. logical points in the computation and provide the basis for an efficient, lightweight coordination mechanism. requirements: the application performs iterative process- We show that many powerful high-level programming ing on a real-time data stream, and supports interac- models can be built on Naiad’s low-level primitives, en- tive queries on a fresh, consistent view of the results. abling such diverse tasks as streaming data analysis, it- However, no existing system satisfies all three require- erative machine learning, and interactive graph mining.
    [Show full text]
  • Tapkee: an Efficient Dimension Reduction Library
    JournalofMachineLearningResearch14(2013)2355-2359 Submitted 2/12; Revised 5/13; Published 8/13 Tapkee: An Efficient Dimension Reduction Library Sergey Lisitsyn [email protected] Samara State Aerospace University 34 Moskovskoye shosse 443086 Samara, Russia Christian Widmer∗ [email protected] Computational Biology Center Memorial Sloan-Kettering Cancer Center 415 E 68th street New York City 10065 NY, USA Fernando J. Iglesias Garcia [email protected] KTH Royal Institute of Technology 79 Valhallavagen¨ 10044 Stockholm, Sweden Editor: Cheng Soon Ong Abstract We present Tapkee, a C++ template library that provides efficient implementations of more than 20 widely used dimensionality reduction techniques ranging from Locally Linear Embedding (Roweis and Saul, 2000) and Isomap (de Silva and Tenenbaum, 2002) to the recently introduced Barnes- Hut-SNE (van der Maaten, 2013). Our library was designed with a focus on performance and flexibility. For performance, we combine efficient multi-core algorithms, modern data structures and state-of-the-art low-level libraries. To achieve flexibility, we designed a clean interface for ap- plying methods to user data and provide a callback API that facilitates integration with the library. The library is freely available as open-source software and is distributed under the permissive BSD 3-clause license. We encourage the integration of Tapkee into other open-source toolboxes and libraries. For example, Tapkee has been integrated into the codebase of the Shogun toolbox (Son- nenburg et al., 2010), giving us access to a rich set of kernels, distance measures and bindings to common programming languages including Python, Octave, Matlab, R, Java, C#, Ruby, Perl and Lua.
    [Show full text]
  • ML Cheatsheet Documentation
    ML Cheatsheet Documentation Team Sep 02, 2021 Basics 1 Linear Regression 3 2 Gradient Descent 21 3 Logistic Regression 25 4 Glossary 39 5 Calculus 45 6 Linear Algebra 57 7 Probability 67 8 Statistics 69 9 Notation 71 10 Concepts 75 11 Forwardpropagation 81 12 Backpropagation 91 13 Activation Functions 97 14 Layers 105 15 Loss Functions 117 16 Optimizers 121 17 Regularization 127 18 Architectures 137 19 Classification Algorithms 151 20 Clustering Algorithms 157 i 21 Regression Algorithms 159 22 Reinforcement Learning 161 23 Datasets 165 24 Libraries 181 25 Papers 211 26 Other Content 217 27 Contribute 223 ii ML Cheatsheet Documentation Brief visual explanations of machine learning concepts with diagrams, code examples and links to resources for learning more. Warning: This document is under early stage development. If you find errors, please raise an issue or contribute a better definition! Basics 1 ML Cheatsheet Documentation 2 Basics CHAPTER 1 Linear Regression • Introduction • Simple regression – Making predictions – Cost function – Gradient descent – Training – Model evaluation – Summary • Multivariable regression – Growing complexity – Normalization – Making predictions – Initialize weights – Cost function – Gradient descent – Simplifying with matrices – Bias term – Model evaluation 3 ML Cheatsheet Documentation 1.1 Introduction Linear Regression is a supervised machine learning algorithm where the predicted output is continuous and has a constant slope. It’s used to predict values within a continuous range, (e.g. sales, price) rather than trying to classify them into categories (e.g. cat, dog). There are two main types: Simple regression Simple linear regression uses traditional slope-intercept form, where m and b are the variables our algorithm will try to “learn” to produce the most accurate predictions.
    [Show full text]
  • 1.2 Le Zhang, Microsoft
    Objective • “Taking recommendation technology to the masses” • Helping researchers and developers to quickly select, prototype, demonstrate, and productionize a recommender system • Accelerating enterprise-grade development and deployment of a recommender system into production • Key takeaways of the talk • Systematic overview of the recommendation technology from a pragmatic perspective • Best practices (with example codes) in developing recommender systems • State-of-the-art academic research in recommendation algorithms Outline • Recommendation system in modern business (10min) • Recommendation algorithms and implementations (20min) • End to end example of building a scalable recommender (10min) • Q & A (5min) Recommendation system in modern business “35% of what consumers purchase on Amazon and 75% of what they watch on Netflix come from recommendations algorithms” McKinsey & Co Challenges Limited resource Fragmented solutions Fast-growing area New algorithms sprout There is limited reference every day – not many Packages/tools/modules off- and guidance to build a people have such the-shelf are very recommender system on expertise to implement fragmented, not scalable, scale to support and deploy a and not well compatible with enterprise-grade recommender by using each other scenarios the state-of-the-arts algorithms Microsoft/Recommenders • Microsoft/Recommenders • Collaborative development efforts of Microsoft Cloud & AI data scientists, Microsoft Research researchers, academia researchers, etc. • Github url: https://github.com/Microsoft/Recommenders • Contents • Utilities: modular functions for model creation, data manipulation, evaluation, etc. • Algorithms: SVD, SAR, ALS, NCF, Wide&Deep, xDeepFM, DKN, etc. • Notebooks: HOW-TO examples for end to end recommender building. • Highlights • 3700+ stars on GitHub • Featured in YC Hacker News, O’Reily Data Newsletter, GitHub weekly trending list, etc.
    [Show full text]
  • Alibaba Amazon
    Online Virtual Sponsor Expo Sunday December 6th Amazon | Science Alibaba Neural Magic Facebook Scale AI IBM Sony Apple Quantum Black Benevolent AI Zalando Kuaishou Cruise Ant Group | Alipay Microsoft Wild Me Deep Genomics Netflix Research Google Research CausaLens Hudson River Trading 1 TALKS & PANELS Scikit-learn and Fairness, Tools and Challenges 5 AM PST (1 PM UTC) Speaker: Adrin Jalali Fairness, accountability, and transparency in machine learning have become a major part of the ML discourse. Since these issues have attracted attention from the public, and certain legislations are being put in place regulating the usage of machine learning in certain domains, the industry has been catching up with the topic and a few groups have been developing toolboxes to allow practitioners incorporate fairness constraints into their pipelines and make their models more transparent and accountable. AIF360 and fairlearn are just two examples available in Python. On the machine learning side, scikit-learn has been one of the most commonly used libraries which has been extended by third party libraries such as XGBoost and imbalanced-learn. However, when it comes to incorporating fairness constraints in a usual scikit- learn pipeline, there are challenges and limitations related to the API, which has made developing a scikit-learn compatible fairness focused package challenging and hampering the adoption of these tools in the industry. In this talk, we start with a common classification pipeline, then we assess fairness/bias of the data/outputs using disparate impact ratio as an example metric, and finally mitigate the unfair outputs and search for hyperparameters which give the best accuracy while satisfying fairness constraints.
    [Show full text]
  • GNU/Linux AI & Alife HOWTO
    GNU/Linux AI & Alife HOWTO GNU/Linux AI & Alife HOWTO Table of Contents GNU/Linux AI & Alife HOWTO......................................................................................................................1 by John Eikenberry..................................................................................................................................1 1. Introduction..........................................................................................................................................1 2. Traditional Artificial Intelligence........................................................................................................1 3. Connectionism.....................................................................................................................................1 4. Evolutionary Computing......................................................................................................................1 5. Alife & Complex Systems...................................................................................................................1 6. Agents & Robotics...............................................................................................................................1 7. Programming languages.......................................................................................................................2 8. Missing & Dead...................................................................................................................................2 1. Introduction.........................................................................................................................................2
    [Show full text]
  • Arxiv:1702.01460V5 [Cs.LG] 10 Dec 2018 1
    Journal of Machine Learning Research 1 (2016) 1-15 Submitted 8/93;; Published 9/93 scikit-multilearn: A scikit-based Python environment for performing multi-label classification Piotr Szyma´nski [email protected],illimites.edu.plg Department of Computational Intelligence Wroc law University of Science and Technology Wroc law, Poland illimites foundation Wroc law, Poland Tomasz Kajdanowicz [email protected] Department of Computational Intelligence Wroc law University of Science and Technology Wroc law, Poland Editor: Leslie Pack Kaelbling Abstract scikit-multilearn is a Python library for performing multi-label classification. The library is compatible with the scikit/scipy ecosystem and uses sparse matrices for all internal op- erations. It provides native Python implementations of popular multi-label classification methods alongside a novel framework for label space partitioning and division. It includes modern algorithm adaptation methods, network-based label space division approaches, which extracts label dependency information and multi-label embedding classifiers. It pro- vides python wrapped access to the extensive multi-label method stack from Java libraries and makes it possible to extend deep learning single-label methods for multi-label tasks. The library allows multi-label stratification and data set management. The implementa- tion is more efficient in problem transformation than other established libraries, has good test coverage and follows PEP8. Source code and documentation can be downloaded from http://scikit.ml and also via pip. The library follows BSD licensing scheme. Keywords: Python, multi-label classification, label-space clustering, multi-label embed- ding, multi-label stratification arXiv:1702.01460v5 [cs.LG] 10 Dec 2018 1.
    [Show full text]
  • Microsoft Recommenders Tools to Accelerate Developing Recommender Systems Scott Graham, Jun Ki Min, Tao Wu {Scott.Graham, Jun.Min, Tao.Wu} @ Microsoft.Com
    Microsoft Recommenders Tools to Accelerate Developing Recommender Systems Scott Graham, Jun Ki Min, Tao Wu {Scott.Graham, Jun.Min, Tao.Wu} @ microsoft.com Microsoft Recommenders repository is an open source collection of python utilities and Jupyter notebooks to help accelerate the process of designing, evaluating, and deploying recommender systems. The repository is actively maintained and constantly being improved and extended. It is publicly available on GitHub and licensed through a permissive MIT License to promote widespread use of the tools. Core components reco_utils • Dataset helpers and splitters • Ranking & rating metrics • Hyperparameter tuning helpers • Operationalization utilities notebooks • Spark ALS • Inhouse: SAR, Vowpal Wabbit, LightGBM, RLRMC, xDeepFM • Deep learning: NCF, Wide-Deep • Open source frameworks: Surprise, FastAI tests • Unit tests • Smoke & Integration tests Workflow Train set Recommender train Environment Data Data split setup load Validation set Hyperparameter tune Operationalize Test set Evaluate • Rating metrics • Content based models • Clone & conda-install on a local / • Chronological split • Ranking metrics • Collaborative-filtering models virtual machine (Windows / Linux / • Stratified split • Classification metrics • Deep-learning models Mac) or Databricks • Non-overlap split • Docker • Azure Machine Learning service (AzureML) • Random split • Tuning libraries (NNI, HyperOpt) • pip install • Azure Kubernetes Service (AKS) • Tuning services (AzureML) • Azure Databricks Example: Movie Recommendation
    [Show full text]
  • Machine Learning for Genomic Sequence Analysis
    Machine Learning for Genomic Sequence Analysis - Dissertation - vorgelegt von Dipl. Inform. S¨oren Sonnenburg aus Berlin Von der Fakult¨at IV - Elektrotechnik und Informatik der Technischen Universit¨at Berlin zur Erlangung des akademischen Grades Doktor der Naturwissenschaften — Dr. rer. nat. — genehmigte Dissertation Promotionsaussschuß Vorsitzender: Prof. Dr. Thomas Magedanz • Berichter: Dr. Gunnar R¨atsch • Berichter: Prof. Dr. Klaus-Robert Muller¨ • Tag der wissenschaftlichen Aussprache: 19. Dezember 2008 Berlin 2009 D83 Acknowledgements Above all, I would like to thank Dr. Gunnar R¨atsch and Prof. Dr. Klaus-Robert M¨uller for their guidance and inexhaustible support, without which writing this thesis would not have been possible. All of the work in this thesis has been done at the Fraunhofer Institute FIRST in Berlin and the Friedrich Miescher Laboratory in T¨ubingen. I very much enjoyed the inspiring atmosphere in the IDA group headed by K.-R. M¨uller and in G. R¨atsch’s group in the FML. Much of what we have achieved was only possible in a joint effort. As the various fruitful within-group collaborations expose — we are a team. I would like to thank Fabio De Bona, Lydia Bothe, Vojtech Franc, Sebastian Henschel, Motoaki Kawanabe, Cheng Soon Ong, Petra Philips, Konrad Rieck, Reiner Schulz, Gabriele Schweikert, Christin Sch¨afer, Christian Widmer and Alexander Zien for reading the draft, helpful discussions and moral support. I acknowledge the support from all members of the IDA group at TU Berlin and Fraunhofer FIRST and the members of the Machine Learning in Computational Biology group at the Friedrich Miescher Laboratory, especially for the tolerance of letting me run the large number of compute-jobs that were required to perform the experiments.
    [Show full text]
  • The Design and Implementation of Low-Latency Prediction Serving Systems
    The Design and Implementation of Low-Latency Prediction Serving Systems Daniel Crankshaw Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2019-171 http://www2.eecs.berkeley.edu/Pubs/TechRpts/2019/EECS-2019-171.html December 16, 2019 Copyright © 2019, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Acknowledgement To my parents The Design and Implementation of Low-Latency Prediction Serving Systems by Daniel Crankshaw A dissertation submitted in partial satisfaction of the requirements for the degree of Doctor of Philosophy in Computer Science in the Graduate Division of the University of California, Berkeley Committee in charge: Professor Joseph Gonzalez, Chair Professor Ion Stoica Professor Fernando Perez Fall 2019 The Design and Implementation of Low-Latency Prediction Serving Systems Copyright 2019 by Daniel Crankshaw 1 Abstract The Design and Implementation of Low-Latency Prediction Serving Systems by Daniel Crankshaw Doctor of Philosophy in Computer Science University of California, Berkeley Professor Joseph Gonzalez, Chair Machine learning is being deployed in a growing number of applications which demand real- time, accurate, and cost-efficient predictions under heavy query load. These applications employ a variety of machine learning frameworks and models, often composing several models within the same application.
    [Show full text]