DOCSLIB.ORG

Amazon Sagemaker Ground Truth 第9回 Amazon Sagemaker 事例祭り #Sagemaker Fes

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Amazon Sagemaker Ground Truth 第9回 Amazon Sagemaker 事例祭り #Sagemaker Fes Amazon SageMaker Ground Truth 第9回 Amazon SageMaker 事例祭り #sagemaker_fes 大渕 麻莉 アマゾン ウェブ サービス ジャパン株式会社 機械学習 ソリューション アーキテクト October 30, 2019 © 2019, Amazon Web Services, Inc. or its Affiliates. 本セッションで話すこと • 機械学習のワークフローとアノテーション (5分) • Amazon SageMaker Ground Truth (15分) • 6種類の組み込みラベリングツールと3種類のワーカー • 自動ラベリングとラベルの統合 • ラベリングジョブの作り方 • Demo (25分) © 2019, Amazon Web Services, Inc. or its Affiliates. 機械学習のワークフロー © 2019, Amazon Web Services, Inc. or its Affiliates. 機械学習の一般的なワークフロー データ収集 クリーン モデルのデプロイ 推論・監視 学習データの生成 アップ 本番環境に データ変換・ デプロイ ラベル付け モデル評価 トレーニング モデルの学習 © 2019, Amazon Web Services, Inc. or its Affiliates. 教師あり学習 入力データ モデル (Neural Network) 予測値 “イヌ” 勾配 損失 © 2019, Amazon Web Services, Inc. or its Affiliates. 機械学習の一般的なワークフロー 教師あり学習では ラベル付け (アノテーション) データ収集 が必要 クリーン 推論・監視 アップ 本番環境に データ変換・ デプロイ ラベル付け モデル評価 トレーニング © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon SageMaker Ground Truth © 2019, Amazon Web Services, Inc. or its Affiliates. ラベル付けの難しさ • 深層学習には大規模なデータセットが必要 • 大量のデータへのラベル付けはしんどい • ラベル付けの精度も求められる • 時間もお金もかかる • ツールの準備 (認証・UI・進捗管理) © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon SageMaker Ground Truth 簡単かつ正確にデータセットのラベル付けができるサービス • アノテーションの一般的なワークフローをサポート • 6種類の組み込みラベリングツールを提供 • アノテーション作業を行うワーカーとの連携・管理機能を提供 • 大規模データセットに対しては自動ラベリング機能で最大70%のコスト削減 迅速なラベル付け ワーカーとの連携が容易 高精度 © 2019, Amazon Web Services, Inc. or its Affiliates. 6種類の組み込みラベリングツール + カスタムジョブ New! カスタム © 2019, Amazon Web Services, Inc. or its Affiliates. カスタムラベリングのワークフロー AWS Lambda AWS Lambda © 2019, Amazon Web Services, Inc. or its Affiliates. 3種類のワーカーが選択可能 • パブリック • Amazon Mechanical Turk で 24時間 x 365日、50万人以上のワーカー • プライベート • 社員などを登録。データを組織内でのみ使用 • ワーカー管理は Amazon Cognito を利用 (OIDC や SAML とも連携可) • ベンダー • AWS Marketplace 登録済みの 3rd パーティーベンダーに依頼 © 2019, Amazon Web Services, Inc. or its Affiliates. アクティブラーニングと自動データラベリング 自動で ラベル付け 入力データセット アクティブ 精度の高い ラーニング 学習用データセット 信頼度の低いデータは 人間がアノテーション 人がラベル付けしたデータから アクティブラーニングのモデルを学習 © 2019, Amazon Web Services, Inc. or its Affiliates. アクティブラーニングと自動データラベリング 自動で ラベル付け 入力データセット アクティブ 精度の高い ラーニング 学習用データセット 信頼度の低いデータは アノテーターの評価は!? 人間がラベル付け © 2019, Amazon Web Services, Inc. or its Affiliates. ラベルの決定 (Label Consolidation): 多数決の場合 bulldog sharpei bulldog bulldog bulldog © 2019, Amazon Web Services, Inc. or its Affiliates. ラベルの決定 (Label Consolidation): 正答率による評価 (犬に詳しい人) 正しい ラベルを 選ぶ確率 0.7 0.9 0.5 0.3 bulldog sharpei bulldog bulldog bulldog 0.1 sharpei 0.9 • 単純な多数決ではなく、ワーカーの正答率をもとにラベルを決定することで アノテーションの質を担保 • 何人のワーカーの結果をまとめるかはジョブ作成時に指定 © 2019, Amazon Web Services, Inc. or its Affiliates. ラベリングジョブの作り方 © 2019, Amazon Web Services, Inc. or its Affiliates. ラベリングジョブ作成の4ステップ 1. データセットの準備 2. タスクの定義 3. ワーカーの選択 4. ラベリングツールの設定 © 2019, Amazon Web Services, Inc. or its Affiliates. 1. データセットの準備 • データと結果を保存するためのS3バケットを用意 • アノテーション対象の画像群をS3バケットに保存 （テキスト分類の場合は Text/CSV ファイル） • 画像群のパスを記述したマニフェストファイルを作成し、S3バケットに保存 （マニフェストファイルは自動生成も可能） input.manifest Amazon S3 JSON Lines フォーマット © 2019, Amazon Web Services, Inc. or its Affiliates. 1. データセットの準備 (マニフェストファイル) input.manifest {"source-ref": "s3://sagemaker-groundtruth-demo/SSDB00001.JPG"} {"source-ref": "s3://sagemaker-groundtruth-demo/SSDB00002.JPG"} {"source-ref": "s3://sagemaker-groundtruth-demo/SSDB00003.JPG"} {"source-ref": "s3://sagemaker-groundtruth-demo/SSDB00004.JPG"} {"source-ref": "s3://sagemaker-groundtruth-demo/SSDB00005.JPG"} {"source-ref": "s3://sagemaker-groundtruth-demo/SSDB00006.JPG"} {"source-ref": "s3://sagemaker-groundtruth-demo/SSDB00007.JPG"} “source-ref” で S3 のパスを指定。 あるいは “source” に直接テキストを書くこともできる © 2019, Amazon Web Services, Inc. or its Affiliates. 2. タスクの定義 • 画像 • 画像分類 • 物体検出 • セマンティック セグメンテーション • ラベルの検証 • テキスト • テキスト分類 • 固有表現抽出 • あるいはユーザ定義の カスタムタスク © 2019, Amazon Web Services, Inc. or its Affiliates. 3. ワーカーの選択 • Public • Amazon Mechanical Turkを利用 • Private • 社員などワーカーを自ら調達 •ベンダー • 3rd パーティーベンダーに依頼 • 追加オプションも選べる • 自動ラベリング • 複数ワーカーによる ラベル付け © 2019, Amazon Web Services, Inc. or its Affiliates. 4. ラベリングツールの設定 • アノテーションの指示書を書く • 良い例・悪い例を記載 • ラベルを設定 © 2019, Amazon Web Services, Inc. or its Affiliates. 出力 (拡張マニフェストファイル) output.manifest { "source-ref": "s3://sagemaker-groundtruth-demo/SSDB00001.JPG", "GroundTruthDemo": { "annotations": [ {"class_id": 0, "width": 54, "top": 482, "height": 39, "left": 337}, {"class_id": 0, "width": 69, "top": 495, "height": 53, "left": 461}, {"class_id": 0, "width": 52, "top": 482, "height": 41, "left": 523} ], "image_size": [{"width": 1280, "depth": 3, "height": 960} ] }, "GroundTruthDemo-metadata": { "job-name": "labeling-job/groundtruthdemo", "class-map": {"0": "Car"}, "human-annotated": "yes", "objects": [ {"confidence": 0.94}, {"confidence": 0.94}, {"confidence": 0.94},], "creation-date": "2018-11-26T04:01:09.038134", © 2019, Amazon"type": Web Services, "groundtruth/objectInc. or its Affiliates. -detection" } } 拡張マニフェストファイルをトレーニングジョブに渡す notebook.ipynb 詳しくはドキュメント「拡張マニフェストファイルを使用してトレーニングジョブにデータセットメタデータを提供する」 © 2019, Amazon Web Services, Inc. or its Affiliates. Demo © 2019, Amazon Web Services, Inc. or its Affiliates. ジョブ一覧 © 2019, Amazon Web Services, Inc. or its Affiliates. ジョブを作成してみる © 2019, Amazon Web Services, Inc. or its Affiliates. ワーカーのログイン後 © 2019, Amazon Web Services, Inc. or its Affiliates. アノテーション画面 © 2019, Amazon Web Services, Inc. or its Affiliates. 出力プレビュー (Augmented Manifest ファイル) © 2019, Amazon Web Services, Inc. or its Affiliates. アノテーションしたデータをビルトインアルゴリズムに © 2019, Amazon Web Services, Inc. or its Affiliates. Amazon SageMaker Ground Truth お客様事例 https://aws.amazon.com/jp/sagemaker/groundtruth/customers/ © 2019, Amazon Web Services, Inc. or its Affiliates. 価格と提供リージョン 価格 • ラベル付けした対象の数に応じた利⽤料 $0.08 / 個 (5万個以上はより安 価) • Amazon Mechanical Turk および 外部ベンダの利⽤料 • ⾃動ラベリング利⽤時は Amazon SageMaker での学習/推論の利⽤料 • 詳細は「Amazon SageMaker Ground Truth の料⾦」 提供リージョン • アジアパシフィック (東京・ソウル・ムンバイ・シンガポール・シド ニー)・カナダ (中部)・EU (フランクフルト・アイルランド・ロンドン)・ ⽶国東部 (バージニア北部・オハイオ)・⽶国⻄部 (オレゴン) © 2019, Amazon Web Services, Inc. or its Affiliates. SageMaker Ground Truth まとめ • データに正解ラベル (Ground Truth) を付与するのためのサービス • 6種類の組み込みラベリングツール • ラベル付け作業を行うワーカー (Public/Private/3rd party) との連携・管理 • 大規模データセットに対しては自動ラベリング機能で最大70%のコスト削減 • Example • Amazon SageMaker Examples にいくつかあります (英語) • 日本語サンプル「SageMaker Ground Truth カスタムラベリング キーポ イント」 • ブログ(固有表現抽出がサポートされる前)「Amazon SageMaker Ground Truth を使ったカスタムデータラベリングワークフローの構築」 © 2019, Amazon Web Services, Inc. or its Affiliates. Thank you! [email protected] © 2019, Amazon Web Services, Inc. or its Affiliates. .
Load more

Recommended publications
  • Building Alexa Skills Using Amazon Sagemaker and AWS Lambda
    Bootcamp: Building Alexa Skills Using Amazon SageMaker and AWS Lambda Description In this full-day, intermediate-level bootcamp, you will learn the essentials of building an Alexa skill, the AWS Lambda function that supports it, and how to enrich it with Amazon SageMaker. This course, which is intended for students with a basic understanding of Alexa, machine learning, and Python, teaches you to develop new tools to improve interactions with your customers. Intended Audience This course is intended for those who: • Are interested in developing Alexa skills • Want to enhance their Alexa skills using machine learning • Work in a customer-focused industry and want to improve their customer interactions Course Objectives In this course, you will learn: • The basics of the Alexa developer console • How utterances and intents work, and how skills interact with AWS Lambda • How to use Amazon SageMaker for the machine learning workflow and for enhancing Alexa skills Prerequisites We recommend that attendees of this course have the following prerequisites: • Basic familiarity with Alexa and Alexa skills • Basic familiarity with machine learning • Basic familiarity with Python • An account on the Amazon Developer Portal Delivery Method This course is delivered through a mix of: • Classroom training • Hands-on Labs Hands-on Activity • This course allows you to test new skills and apply knowledge to your working environment through a variety of practical exercises • This course requires a laptop to complete lab exercises; tablets are not appropriate Duration One Day Course Outline This course covers the following concepts: • Getting started with Alexa • Lab: Building an Alexa skill: Hello Alexa © 2018, Amazon Web Services, Inc.
    [Show full text]
  • Innovation on Every Front
    INNOVATION ON EVERY FRONT AWS DeepComposer uses AI to compose music. AI AWS DeepComposer The new service gives developers a creative way to get started and become familiar with machine learning capabilities. Amazon Transcribe Medical is a machine AI Amazon Transcribe Medical learning service that makes it easy to quickly create accurate transcriptions from medical consultations between patients and physician dictated notes, practitioner/patient consultations, and tele-medicine are automatically converted from speech to text for use in clinical documen- tation applications. Amazon Rekognition Custom Labels helps users AI Amazon Rekognition Custom identify the objects and scenes in images that are specific to business needs. For example, Labels users can find corporate logos in social media posts, identify products on store shelves, classify machine parts in an assembly line, distinguish healthy and infected plants, or detect animated characters in videos. Amazon SageMaker Model Monitor is a new AI Amazon SageMaker Model capability of Amazon SageMaker that automatically monitors machine learning (ML) Monitor models in production, and alerts users when data quality issues appear. Amazon SageMaker is a fully managed service AI Amazon SageMaker that provides every developer and data scientist with the ability to build, train, and deploy ma- Experiments chine learning (ML) models quickly. SageMaker removes the heavy lifting from each step of the machine learning process to make it easier to develop high quality models. Amazon SageMaker Debugger is a new capa- AI Amazon SageMaker Debugger bility of Amazon SageMaker that automatically identifies complex issues developing in machine learning (ML) training jobs. Amazon SageMaker Autopilot automatically AI Amazon SageMaker Autopilot trains and tunes the best machine learning models for classification or regression, based on your data while allowing to maintain full control and visibility.
    [Show full text]
  • Build Your Own Anomaly Detection ML Pipeline
    Device telemetry data is ingested from the field Build your own Anomaly Detection ML Pipeline 1 devices on a near real-time basis by calls to the This end-to-end ML pipeline detects anomalies by ingesting real-time, streaming data from various network edge field devices, API via Amazon API Gateway. The requests get performing transformation jobs to continuously run daily predictions/inferences, and retraining the ML models based on the authenticated/authorized using Amazon Cognito. incoming newer time series data on a daily basis. Note that Random Cut Forest (RCF) is one of the machine learning algorithms Amazon Kinesis Data Firehose ingests the data in for detecting anomalous data points within a data set and is designed to work with arbitrary-dimensional input. 2 real time, and invokes AWS Lambda to transform the data into parquet format. Kinesis Data AWS Cloud Firehose will automatically scale to match the throughput of the data being ingested. 1 Real-Time Device Telemetry Data Ingestion Pipeline 2 Data Engineering Machine Learning Devops Pipeline Pipeline The telemetry data is aggregated on an hourly 3 3 basis and re-partitioned based on the year, month, date, and hour using AWS Glue jobs. The Amazon Cognito AWS Glue Data Catalog additional steps like transformations and feature Device #1 engineering are performed for training the SageMaker AWS CodeBuild Anomaly Detection ML Model using AWS Glue Notebook Create ML jobs. The training data set is stored on Amazon Container S3 Data Lake. Amazon Kinesis AWS Lambda S3 Data Lake AWS Glue Device #2 Amazon API The training code is checked in an AWS Gateway Data Firehose 4 AWS SageMaker CodeCommit repo which triggers a Machine Training CodeCommit Containers Learning DevOps (MLOps) pipeline using AWS Dataset for Anomaly CodePipeline.
    [Show full text]
  • Cloud Computing & Big Data
    Cloud Computing & Big Data PARALLEL & SCALABLE MACHINE LEARNING & DEEP LEARNING Ph.D. Student Chadi Barakat School of Engineering and Natural Sciences, University of Iceland, Reykjavik, Iceland Juelich Supercomputing Centre, Forschungszentrum Juelich, Germany @MorrisRiedel LECTURE 11 @Morris Riedel @MorrisRiedel Big Data Analytics & Cloud Data Mining November 10, 2020 Online Lecture Review of Lecture 10 –Software-As-A-Service (SAAS) ▪ SAAS Examples of Customer Relationship Management (CRM) Applications (PAAS) (introducing a growing range of machine (horizontal learning and scalability artificial enabled by (IAAS) intelligence Virtualization capabilities) in Clouds) [5] AWS Sagemaker [4] Freshworks Web page [3] ZOHO CRM Web page modfied from [2] Distributed & Cloud Computing Book [1] Microsoft Azure SAAS Lecture 11 – Big Data Analytics & Cloud Data Mining 2 / 36 Outline of the Course 1. Cloud Computing & Big Data Introduction 11. Big Data Analytics & Cloud Data Mining 2. Machine Learning Models in Clouds 12. Docker & Container Management 13. OpenStack Cloud Operating System 3. Apache Spark for Cloud Applications 14. Online Social Networking & Graph Databases 4. Virtualization & Data Center Design 15. Big Data Streaming Tools & Applications 5. Map-Reduce Computing Paradigm 16. Epilogue 6. Deep Learning driven by Big Data 7. Deep Learning Applications in Clouds + additional practical lectures & Webinars for our hands-on assignments in context 8. Infrastructure-As-A-Service (IAAS) 9. Platform-As-A-Service (PAAS) ▪ Practical Topics 10. Software-As-A-Service
    [Show full text]
  • Build a Secure Enterprise Machine Learning Platform on AWS AWS Technical Guide Build a Secure Enterprise Machine Learning Platform on AWS AWS Technical Guide
    Build a Secure Enterprise Machine Learning Platform on AWS AWS Technical Guide Build a Secure Enterprise Machine Learning Platform on AWS AWS Technical Guide Build a Secure Enterprise Machine Learning Platform on AWS: AWS Technical Guide Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Build a Secure Enterprise Machine Learning Platform on AWS AWS Technical Guide Table of Contents Abstract and introduction .................................................................................................................... i Abstract .................................................................................................................................... 1 Introduction .............................................................................................................................. 1 Personas for an ML platform ............................................................................................................... 2 AWS accounts .................................................................................................................................... 3 Networking architecture .....................................................................................................................
    [Show full text]
  • Analytics Lens AWS Well-Architected Framework Analytics Lens AWS Well-Architected Framework
    Analytics Lens AWS Well-Architected Framework Analytics Lens AWS Well-Architected Framework Analytics Lens: AWS Well-Architected Framework Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Analytics Lens AWS Well-Architected Framework Table of Contents Abstract ............................................................................................................................................ 1 Abstract .................................................................................................................................... 1 Introduction ...................................................................................................................................... 2 Definitions ................................................................................................................................. 2 Data Ingestion Layer ........................................................................................................... 2 Data Access and Security Layer ............................................................................................ 3 Catalog and Search Layer ...................................................................................................
    [Show full text]
  • All Services Compute Developer Tools Machine Learning Mobile
    AlL services X-Ray Storage Gateway Compute Rekognition d Satellite Developer Tools Amazon Sumerian Athena Machine Learning Elastic Beanstalk AWS Backup Mobile Amazon Transcribe Ground Station EC2 Servertess Application EMR Repository Codestar CloudSearch Robotics Amazon SageMaker Customer Engagement Amazon Transtate AWS Amplify Amazon Connect Application Integration Lightsail Database AWS RoboMaker CodeCommit Management & Governance Amazon Personalize Amazon Comprehend Elasticsearch Service Storage Mobile Hub RDS Amazon Forecast ECR AWS Organizations Step Functions CodeBuild Kinesis Amazon EventBridge AWS Deeplens Pinpoint S3 AWS AppSync DynamoDe Amazon Textract ECS CloudWatch Blockchain CodeDeploy Quicksight EFS Amazon Lex Simple Email Service AWS DeepRacer Device Farm ElastiCache Amazont EKS AWS Auto Scaling Amazon Managed Blockchain CodePipeline Data Pipeline Simple Notification Service Machine Learning Neptune FSx Lambda CloudFormation Analytics Cloud9 AWS Glue Simple Queue Service Amazon Polly Business Applications $3 Glacer AR & VR Amazon Redshift SWF Batch CloudTrail AWS Lake Formation Server Migration Service lot Device Defender Alexa for Business GuardDuty MediaConnect Amazon QLDB WorkLink WAF & Shield Config AWS Well. Architected Tool Route 53 MSK AWS Transfer for SFTP Artifact Amazon Chime Inspector lot Device Management Amazon DocumentDB Personal Health Dashboard C MediaConvert OpsWorks Snowball API Gateway WorkMait Amazon Macie MediaLive Service Catalog AWS Chatbot Security Hub Security, Identity, & Internet of Things loT
    [Show full text]
  • Predictive Analytics with Amazon Sagemaker
    Predictive Analytics with Amazon SageMaker Steve Shirkey Specialist SA, AWS (Singapore) © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. 2 Explosion in AI and ML Use Cases Image recognition and tagging for photo organization Object detection, tracking and navigation for Autonomous Vehicles Speech recognition & synthesis in Intelligent Voice Assistants Algorithmic trading strategy performance improvement Sentiment analysis for targeted advertisements © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. ~1997 © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Thousands of Amazon Engineers Focused on Machine Learning Fulfillment & Search & Existing New At logistics discovery products products AWS © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. © 2018, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Over 20 years of AI at Amazon… • Applied research • Q&A systems • Core research • Supply chain optimization • Alexa • Advertising • Demand forecasting • Machine translation • Risk analytics • Video content analysis • Search • Robotics • Recommendations • Lots of computer vision… • AI services • NLP/NLU © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. ML @ AWS Put machine learning in the OUR MISSION hands of every developer and data scientist © 2018 Amazon Web Services, Inc. or its Affiliates. All rights reserved. Customers Running Machine Learning On AWS Today © 2018 Amazon Web Services, Inc. or its Affiliates.
    [Show full text]
  • Fraud Detection in Online Lending Using Amazon Sagemaker
    Case study Fraud Detection in online lending using Amazon SageMaker A powerful Machine Learning-based model enabled this company to implement an automated anomaly detection and fraud prevention mechanism that cut online Payment frauds by 27% ABOUT AXCESS.IO ABOUT Lend.in AXCESS.IO offers world-class Managed Cloud Services to businesses Designed for today's financial enterprises, Lend. It provides a worldwide. In a relatively short period, AXCESS.IO has already low-code, digital-first lending platform that accelerates scalable served several enterprise clients and has quickly evolved into a digital transformation, improves speed to market, and supports niche consulting firm specializing in Cloud Advisory, Cloud Managed business growth. The company helps many banks, credit Services, and DevOps Automation. institutions and lending agencies modernize their loan processes. The Lend-in platform leverages user-friendly API-based integrations and powerful cogn axcess.io © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved. THE CHALLENGE Lend.in’s lending management platform is leveraged by a number of financial services companies to enable digital lending. Through this platform, the company’s customers take advantage of a highly simplified application process where they can apply for credit using any device from anywhere. This unique offering provides much-needed credit that enhances the experience. However, there was no way to accurately screen applicants’ profile and compare against the pattern, so it also increases the potential for fraud. This issue was seriously increasing the credit risk, as well as it was a gap in Lend.in’s standing as an integrated FinTech solutions company THE SOLUTION Lend.
    [Show full text]
  • Enabling Continual Learning in Deep Learning Serving Systems
    ModelCI-e: Enabling Continual Learning in Deep Learning Serving Systems Yizheng Huang∗ Huaizheng Zhang∗ Yonggang Wen Nanyang Technological University Nanyang Technological University Nanyang Technological University [email protected] [email protected] [email protected] Peng Sun Nguyen Binh Duong TA SenseTime Group Limited Singapore Management University [email protected] [email protected] ABSTRACT streamline model deployment and provide optimizing mechanisms MLOps is about taking experimental ML models to production, (e.g., batching) for higher efficiency, thus reducing inference costs. i.e., serving the models to actual users. Unfortunately, existing ML We note that existing DL serving systems are not able to handle serving systems do not adequately handle the dynamic environ- the dynamic production environments adequately. The main reason ments in which online data diverges from offline training data, comes from the concept drift [8, 10] issue - DL model quality is resulting in tedious model updating and deployment works. This tied to offline training data and will be stale soon after deployed paper implements a lightweight MLOps plugin, termed ModelCI-e to serving systems. For instance, spam patterns keep changing (continuous integration and evolution), to address the issue. Specif- to avoid detection by the DL-based anti-spam models. In another ically, it embraces continual learning (CL) and ML deployment example, as fashion trends shift rapidly over time, a static model techniques, providing end-to-end supports for model updating and will result in an inappropriate product recommendation. If a serving validation without serving engine customization. ModelCI-e in- system is not able to quickly adapt to the data shift, the quality of cludes 1) a model factory that allows CL researchers to prototype inference would degrade significantly.
    [Show full text]
  • Media2cloud Implementation Guide Media2cloud Implementation Guide
    Media2Cloud Implementation Guide Media2Cloud Implementation Guide Media2Cloud: Implementation Guide Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. Media2Cloud Implementation Guide Table of Contents Home ............................................................................................................................................... 1 Overview ........................................................................................................................................... 2 Cost .......................................................................................................................................... 2 Architecture ............................................................................................................................... 2 Ingest Workflow ................................................................................................................. 3 Analysis Workflow .............................................................................................................. 3 Labeling Workflow ............................................................................................................
    [Show full text]
  • Aws-Mlops-Framework.Pdf
    AWS MLOps Framework Implementation Guide AWS MLOps Framework Implementation Guide AWS MLOps Framework: Implementation Guide Copyright © Amazon Web Services, Inc. and/or its affiliates. All rights reserved. Amazon's trademarks and trade dress may not be used in connection with any product or service that is not Amazon's, in any manner that is likely to cause confusion among customers, or in any manner that disparages or discredits Amazon. All other trademarks not owned by Amazon are the property of their respective owners, who may or may not be affiliated with, connected to, or sponsored by Amazon. AWS MLOps Framework Implementation Guide Table of Contents Home ............................................................................................................................................... 1 Cost ................................................................................................................................................ 3 Example cost table ..................................................................................................................... 3 Architecture overview ........................................................................................................................ 4 Template option 1: Single account deployment .............................................................................. 4 Template option 2: Multi-account deployment ............................................................................... 5 Shared resources and data between accounts ...............................................................................
    [Show full text]