Seldon Deploy smv Sep 02, 2021 CONTENTS 1 About 1 1.1 What is Seldon-Deploy?.........................................1 1.2 Seldon Core...............................................1 1.3 KFServing................................................3 1.4 Alibi...................................................3 1.4.1 Audience............................................4 1.4.1.1 Who should use Seldon Deploy?...........................4 2 Release Notes 7 2.1 Seldon Deploy 1.3.0...........................................7 2.1.1 What’s New...........................................7 2.1.2 Bug fixes............................................7 2.2 Seldon Deploy 1.2.1...........................................8 2.2.1 Bug fixes............................................8 2.3 Seldon Deploy 1.2.0...........................................8 2.3.1 Breaking Changes........................................8 2.3.2 What’s New...........................................8 2.3.3 Bug fixes............................................8 2.4 Seldon Deploy 1.1.2...........................................8 2.4.1 Bug fixes............................................9 2.5 Seldon Deploy 1.1.0...........................................9 2.5.1 What’s New...........................................9 2.5.2 Bug fixes............................................9 2.6 Seldon Deploy 1.0.0...........................................9 2.6.1 Breaking Changes........................................9 2.6.2 What’s New...........................................9 2.6.3 Bug fixes............................................ 10 2.7 Seldon Deploy 0.9.0........................................... 10 2.7.1 Breaking Changes........................................ 10 2.7.2 What’s New........................................... 10 2.7.3 Bug fixes............................................ 10 2.8 Seldon Deploy 0.8.2........................................... 10 2.8.1 Bug fixes............................................ 10 2.9 Seldon Deploy 0.8.1........................................... 11 2.9.1 What’s New........................................... 11 2.9.2 Bug fixes............................................ 11 2.10 Seldon Deploy 0.8.0........................................... 11 2.10.1 Breaking Changes........................................ 11 2.10.2 What’s New........................................... 11 2.10.3 Bug fixes............................................ 12 i 2.11 Seldon Deploy 0.7.0........................................... 12 2.11.1 Breaking Changes........................................ 12 2.11.2 What’s New........................................... 12 2.11.3 Bug fixes............................................ 12 2.12 Seldon Deploy 0.6.2........................................... 12 2.12.1 Bug fixes............................................ 12 2.13 Seldon Deploy 0.6.1........................................... 12 2.13.1 Bug fixes............................................ 13 2.14 Seldon Deploy 0.6.0........................................... 13 2.14.1 Breaking Changes........................................ 13 2.14.2 What’s New........................................... 13 2.14.3 Bug fixes............................................ 13 2.15 Seldon Deploy 0.5.2........................................... 14 2.15.1 Bug fixes............................................ 14 3 Getting Started 15 3.1 Installation Types............................................. 15 3.2 Component integrations......................................... 15 3.3 Pre-requisites............................................... 16 3.4 Cluster Requirements.......................................... 16 3.5 Older Kubernetes Versions........................................ 16 3.6 Newer Kubernetes Versions....................................... 17 3.7 References................................................ 17 3.7.1 Download resources...................................... 17 3.7.2 Applying a License....................................... 17 3.7.3 Trial Installation........................................ 18 3.7.3.1 Trial Installation Options............................... 18 3.7.3.2 Configuration..................................... 18 3.7.3.3 References....................................... 19 3.7.4 Production Installation..................................... 26 3.7.4.1 Components overview................................. 26 3.7.4.2 References....................................... 27 3.7.5 Multi-tenant Installation.................................... 70 3.7.5.1 Multi-tenant Guidance................................. 71 3.7.6 OpenShift Installation...................................... 73 3.7.6.1 Red Hat Marketplace Installation........................... 73 3.7.6.2 Other OpenShift Installations............................. 73 3.7.7 Installation Types........................................ 74 3.7.7.1 Trial vs Production Install............................... 74 3.7.7.2 Easiest Ways to Get Started.............................. 74 3.7.7.3 What if I also want an ML training platform?..................... 75 3.7.7.4 How to Size a Cluster?................................ 75 4 Product Tour 77 4.1 Deployment Dashboard......................................... 77 4.1.1 Request Monitoring....................................... 78 4.1.2 Distributions Monitoring.................................... 78 4.1.3 Request Logs.......................................... 80 4.1.4 Requests to Models....................................... 81 4.1.4.1 Make a prediction................................... 81 4.1.4.2 Load Test....................................... 82 4.2 Deployment Wizard........................................... 82 4.2.1 Seldon Deployment Wizard................................... 83 4.2.2 Kubeflow Inference Service Wizard.............................. 83 ii 4.3 Model Explanations........................................... 84 4.3.1 Available Methods....................................... 84 4.3.1.1 Tabular Data...................................... 84 4.3.1.2 Text Data........................................ 85 4.3.1.3 Image Data....................................... 86 4.4 Rollout Strategies............................................ 86 4.4.1 Canary.............................................. 86 4.4.2 Shadow............................................. 88 4.5 REST API................................................ 89 4.5.1 Usage.............................................. 90 4.5.2 Authentication......................................... 91 4.5.3 Versioning............................................ 93 4.5.4 Reference............................................ 93 4.5.4.1 API Reference (v1alpha1)............................... 93 5 Demos 95 5.1 Seldon Core Demos........................................... 95 5.1.1 Canary Promotion........................................ 95 5.1.1.1 Iris Model....................................... 95 5.1.1.2 Deploy Model..................................... 96 5.1.1.3 Start Load Test..................................... 96 5.1.1.4 Create Canary..................................... 98 5.1.2 Model Explanations with Anchor Images........................... 99 5.1.2.1 Create Model..................................... 100 5.1.2.2 Get Predictions.................................... 100 5.1.2.3 Add an Anchor Images Explainer........................... 100 5.1.2.4 Get Explanation for one Request........................... 100 5.1.3 Model Explanations with Anchor Tabular........................... 100 5.1.3.1 Create Model..................................... 101 5.1.3.2 Get Predictions.................................... 101 5.1.3.3 Add an Anchor Tabular Explainer........................... 102 5.1.3.4 Get Explanation for one Request........................... 102 5.1.4 Model Explanations with Anchor Text............................. 103 5.1.4.1 Create Model..................................... 103 5.1.4.2 Get Predictions.................................... 104 5.1.4.3 Add an Anchor Text Explainer............................ 104 5.1.4.4 Get Explanation for one Request........................... 104 5.1.5 Model Catalogue........................................ 104 5.1.5.1 Registering Models and Editing Metadata...................... 104 5.1.5.2 Register New Model.................................. 105 5.1.5.3 Edit Model Metadata................................. 105 5.1.6 Outlier Detection with CIFAR10 Image Classifier....................... 106 5.1.6.1 Create Model..................................... 107 5.1.6.2 Setup Outlier detector................................. 107 5.1.6.3 Make Predictions................................... 107 5.1.6.4 View outliers on the Requests Screen......................... 107 5.1.6.5 Monitor outliers on the Monitor Screen........................ 107 5.1.6.6 Troubleshooting.................................... 107 5.1.7 Drift Detection with CIFAR10 Image Classifier........................ 108 5.1.7.1 Create A Model.................................... 108 5.1.7.2 Add A Drift Detector................................. 108 5.1.7.3 Make Predictions................................... 109 5.1.7.4 Monitor Drift On The Monitor Screen........................ 109 5.1.7.5 Troubleshooting.................................... 109 iii 5.1.8 Model Accuracy Metrics with Iris Classifier.......................... 110 5.1.8.1 Create Model..................................... 110 5.1.8.2 Setup Metrics Server................................. 110 5.1.8.3 Make Predictions..................................