AWS Autotech Forum 2020 Online #1

2020/08/07

) (

(

AWS Glue Amazon Kinesis Amazon S3

• •

• • AWS • Next Action

• • • ü ü • Next Step •

LIDAR

Camera

Telematics Control Unit

GPS/HD Maps

Accelerometer/Gyroscope

Odometry

Cellular/DSRC

Radar

Infrared

Ultrasonic

), (, :,

{ “trip_id”: “00000001”, “timestamp”: “202008071000”, “x_value”: “0.342264”, “y_value”: “0.011724”, ”z_value”: “0.924651”, }

/ : , etc.

{ “trip_id”: “00000001”, “timestamp”: “202008071000”, “x_value”: “0.342264”, “y_value”: “0.011724”, ”z_value”: “0.924651”, ( ( )/.,( ( ( }

: c c e

•

• (

•

• • •

1 A

3Hadoop 3

Amazon Simples Storage Service(S3)

• RDBMS

•

• )API( API

Amazon Simple Storage Service (S3) 0934829 0

% 0 PUT 5

1. GET .$$GB 9 Amazon S3 6. API/908

BI IoT

Amazon Amazon EMR Amazon Amazon QuickSight Athena Redshift AWS IoT

AI/ML Amazon Amazon Amazon S3 EC2 RDS

Amazon Lex Amazon SageMaker

Amazon AWS EBS Storage Gateway

Amazon Amazon Polly Amazon Amazon S3 Glacier Rekognition Redshift

Point

API

)

Amazon S3

2 132 -2-

-.2 .2-. 2

2 -.2 12

Amazon Athena User Amazon S3 Amazon QuickSight E Raw

() Amazon S3 A .S S

AWS Lambda

User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth Amazon DocumentDB

• (. ü ( ü ü ü

• ü

Amazon Athena User Amazon S3 Amazon QuickSight E Raw

() Amazon S3 A .S S

AWS Lambda

User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth Amazon DocumentDB

Amazon S3((*1)) SQL

(*1)Federated Query

Amazon Athena User Amazon S3 Amazon QuickSight ) Raw

AWS Lambda Amazon S3 )

(. AWS Lambda

User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth Amazon DocumentDB

)

Amazon Athena User Amazon S3 Amazon QuickSight . E Raw

AWS Lambda Amazon S3 ( .

AWS Lambda

User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth Amazon DocumentDB

Amazon Athena User .. Amazon S3 Amazon QuickSight ( .) Raw

AWS Lambda Amazon S3 . .) (

AWS Lambda

.E User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth Amazon DocumentDB

)

. (

Amazon Athena User Amazon S3 Amazon QuickSight Raw SIM AWS Lambda AWS IoT Core Amazon Amazon S3 Kinesis Data Firehose

AWS Lambda

E User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth E Amazon DocumentDB

AWS IoT CoreIoT

Amazon S3Amazon RedshiftAmazon Elasticsearch Service ,

Amazon Athena User .. Amazon S3 Amazon QuickSight ( .) (

AWS Lambda Amazon S3 . .) (

AWS Lambda

.E User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth Amazon DocumentDB

LcDD Da

d DE

Amazon Redshift User DD Amazon S3 Amazon QuickSight B DA d Raw

AWS Glue Amazon S3 WD A B W

WE ().)

DDbS User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth S e .).

BI , ,

Amazon S3

DWH

OLTP ERP CRM LOB

AWS Lambda AWS Glue AWS Glue Python Shell Spark Job ( Python PySpark 5 Scala NumPySciPyPandas

• SQL • ( SQL • • •

Amazon Athena Amazon Redshift

(

Amazon Athena User (( Amazon S3 Amazon QuickSight 3( (( Raw

AWS Lambda Amazon S3 E( 3( E

E AWS Lambda

( User Amazon S3 Amazon SageMaker Amazon S3 Amazon SageMaker Ground Truth E Amazon DocumentDB

• ü ü

• ü ü ü ü ü ü

Amazon S3

VISION SPEECH TEXT SEARCH CHATBOTS PERSONALIZE FORECASTING FRAUD DEV CONTACT CENTER

Amazon Amazon Amazon Amazon Amazon Amazon Amazon Amazon Amazon Amazon Amazon Fraud Amazon Contact Lens Rekognition Polly Transcribe Comprehend Translate Textract Kendra Lex Personalize Forecast Detector CodeGuru for Amazon +Medical +Medical Connect

ML SERVICES

Model AWS Buil-in Model Model Notebooks Experiments Processing Training & Debugger Autopilot Ground Truth Augmented AI Marketplac e Neo Algorithms Hosting Monitor HPO for ML

Amazon SageMaker

SageMaker Studio IDE

ML FRAMEWORKS & INFRASTRUCTURE

Deep Learning GPUs & CPUs Elastic Inference Inferentia FPGA AMIs/Containers DeepGraphLibrar y

ML SERVICES

Model AWS Buil-in Model Model Notebooks Experiments Processing Training & Debugger Autopilot Ground Truth Augmented AI Marketplac e Neo Algorithms Hosting Monitor HPO for ML

Amazon SageMaker

SageMaker Studio IDE

ML FRAMEWORKS & INFRASTRUCTURE

Deep Learning GPUs & CPUs Elastic Inference Inferentia FPGA AMIs/Containers DeepGraphLibrar y

341 W432.21

A A S

• II Amazon SageMaker • I P Ground Truth •

• II Amazon SageMaker P • PIAI Notebook Instance

• II One Click Training & • I Auto HPO •

• II One Click • Deployment • U

•

1 ,B

5 0

B 5B TB B2015

: https://youtu.be/9fH3y5p8ewE PDF : https://d1.awsstatic.com/events/reinvent/2019/Creating_a_data-driven,_cloud-native_ecosystem_at_BMW_Group_AUT306.pdf © 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. BMW - "Cloud Data Hub"

Data Provider Mask PII Cloud Data Hub InfoSec

Kinesis Private VPC Firehose Source S3

Pull KMS CMK

Legacy Fargate telemetry On-premise SQS SNS

Push Prep Prep S3

Cloud-native telemetry

Local Glue DC Global Glue DC Sync Glue DC

: https://youtu.be/9fH3y5p8ewE PDF : https://d1.awsstatic.com/events/reinvent/2019/Creating_a_data-driven,_cloud-native_ecosystem_at_BMW_Group_AUT306.pdf

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. AWS CMS Telemetry data MQTT (near-real-time) Connected vehicle data ingestion Connected vehicles Telemetry Anomaly detection & alerting Data visualization data JSON Personas

Rule 2

Raw data

Aggregated telemetry data (JSON) – Rule 1 Anomaly table Manufacturers Dealers Consumers Anomalous Data AWS IoT Core Telemetry and DTCs (JSON)—Rule 2 Search

Authentication Device gateway Rules engine Trip aggregation processing Web UI Mobile app Aggregated telemetry data Lifecycle management JSON Rule 3 Trip aggregation table User Aggregated telemetry Notifications pool data [ ignition status ] Driver safety score processing JSON

Rule 4

Safety score alerts OTA images

Geolocation data Microservices JSON Location-based services Rule 5 Logs Asset library Provisioning Fleet console templates Asset information

Services Table Web UI OTA User management pool

© 2020, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Autonomous Driving Data Lake Build an MDF4/Rosbag-based data ingestion and processing pipeline for Autonomous Driving and Advanced Driver Assistance Systems (ADAS).

Ingest data from autonomous fleet 7 1 2 with AWS Outposts for local data processing.

Ingest vehicle telemetry data in real 2 time using AWS IoT Core and Amazon Kinesis Data Firehose. 4 10 3 Remove and transform low quality data.

Schedule the extract, transform, load 4 (ETL) jobs using Apache Airflow. 6 3 Enrich data with weather conditions 5 based on GPS location and timestamp. 6 8 9 Extract metadata using ASAM 1 OpenSCENARIO and store data in Amazon DynamoDB and Amazon Elasticsearch Service. 7 Store data lineage in Amazon 5 Neptune and catalog data using AWS Glue Data Catalog.

8 Process drive data and perform deep signal validation.

9 Perform automated labeling using Amazon SageMaker Ground Truth.

•

• https://github.com/aws-samples/amazon-s3-datalake- handson/blob/master/JP/README.md

• AWS BlackBelt Online

Ø https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/ Ø Amazon S3, Amazon Athena, Amazon QuickSight, Amazon SageMaker(Basic)

• AWS Summit Online 2020

•

• https://github.com/aws-samples/amazon-s3-datalake- handson/blob/master/JP/README.md

• AWS BlackBelt Online

Ø https://aws.amazon.com/jp/aws-jp-introduction/aws-jp-webinar-service-cut/ Ø Amazon S3, Amazon Athena, Amazon QuickSight, Amazon SageMaker(Basic)

• AWS Summit Online 2020 AWS

23 • 13 • .

• Amazon S3 3

• • 1

URL https://bit.ly/2DykTzF