Microsoft Azure in Autonomous Driving ’s approach to automotive

We complement OEMs and suppliers – not compete

We ensure your data is always under your control

We guarantee that the brand and customer experience belongs to you

OEM System Engineering Process

Engineering Development Product Validation Manufacturing & Service

Feasibility Operations Regional Changes Retirement/ Study/concept and Architecture (s) and Upgrades replacement Exploration Maintenance System Validation Plan Lifecycle Processes Concepts of System Operations Validation

System Verification Plan (System Acceptance) System System Verification & Requirements Deployment Subsystem Verification Plan (Subsystem Acceptance) High-Level Subsystem Design Verification Unit/Device Test Plan Unit/Device Detailed Design Testing Document/Approval

Software/Hardware Development Field Installation

Implementation Development Processes Time Line Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)

Replay Tag Performance Simulate Hardware Simulation in the loop

Render/Convert Test-Drive

DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE • Cloud and Analytics partner for • Open source AD Platform • OpenADx ACM, an autonomous and smart • 200+ Member Consortium • Interoperable Eclipse Framework mobility test facility • Microsoft is the cloud provider for • Leveraged by all OEMs, tier ones and Project Apollo worldwide with the technology start ups exception of China • Engaged with ACM to influence standards

Industry

Academia Government

LG’s autonomous vehicle program had unique requirements, including portability, security, and fast turnaround time; Data Box Disk was the perfect solution.

“We needed a way to transfer massive amounts of data for our autonomous vehicle projects, based all around the world. The solution needed to be portable, simple to use, cost-effective and, of course, very secure. The Azure Data Box Disk met all of those criteria.” – Hyoyuel (Andy) Kim, Senior Manager, LG Electronics.

Products and services Organization size Industry Country Azure Data Box Disk 91,000 Automotive Worldwide Azure Blob Storage electronics Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)

Replay Tag Performance Simulate Hardware Simulation in the loop

Render/Convert Test-Drive

DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE

Azure Express Route Azure Compute Databricks & HDInsight Azure IoT Edge

Azure Storage, Data Lake Gen2 Azure Batch AI Azure Machine Learning Container and Kubernetes , Visual Studio Team Services Azure Data Box Cycle Cloud Cognitive Services Azure Key Vault, AAD SOFTWARE TOOLS AND DEVOPS PLATFORM

CI/CD flows Machine Learning Azure Container Source Container Registry Control Service

DATA INGEST

Data Factory

Virtual machine

Cycle Cloud HDInsight, Azure GPU Azure VM Cosmos DB In-car Databricks Logic App data Async storage upload Kubernetes Azure blob Service Simplygon storage (cool, ML PowerBI archive) CosmosDB Data Fast Advanced HDInsight reduction, upload Stream Analytics Batch Analytics Databricks Service Bus redaction IoT Edge ADLS Gen 2 and Azure blob storage Hot Tier Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)

Replay Tag Performance Simulate Hardware Simulation in the loop

Render/Convert Test-Drive

DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Offline Data Transfer Online Data Transfer

Data Box Data Box Disk Data Box Heavy Data Box Gateway Data Box Edge • Capacity: 100 TB • Capacity: 8TB ea.; 40TB/order • Capacity: 1000TB • Virtual device provisioned in • Local Cache Capacity: ~12 TB • Weight: ~50 lbs • Secure, ruggedized USB • Weight 500+ lbs your hypervisor • Includes Data Box Gateway • Secure, ruggedized drives orderable in packs of • Secure, ruggedized • Supports storage gateway, and Azure IoT Edge. appliance 5 (up to 40TB). appliance SMB, NFS, Azure blob, files • Currently in Preview • Currently in Preview • Currently in preview • Currently in preview • Data Box enables bulk migration to Azure when • Perfect for projects that • Same service as Data Box, • Virtual network transfer • Data Box Edge manages network isn’t an option. require a smaller form factor, but targeted to petabyte- appliance (VM), runs on your uploads to Azure and can e.g., autonomous vehicles. sized datasets. choice of hardware. pre-process data prior to upload.

Network Data Transfer Edge Compute

Order Send Fill Return Upload Cloud to Edge Edge to Cloud Pre-processing ML Inferencing Massive Data Ingest – ExpressRoute Ingest up to PB of data daily with Azure Networking Express Route Service

Private, high speed network connections

Connecting your data center and Azure

Predictable performance and Secure

100 + Express Route partners

Data Storage – Azure Blob Storage Store up to Exabytes of data in massively scalable object storage for unstructured data

• Foundational service for Microsoft Scalable • >40 million transactions per second • Multi-PB accounts

• 50+Gbps account throughput Performant • Continued enhancements under Frequently Less frequently Rarely development accessed data accessed data accessed data Secure & • Client & Service Encryption • AAD Integration + ACLs Lower capacity Lowest transaction Lowest capacity Compliant • Broad & deep compliance portfolio cost cost cost • Multiple redundancy options Immediate (ms) Immediate (ms) Hours Durable • Strong consistency, data integrity • Policy: Versioning & WORM locks

Cost • Integrated storage tiers • Lifecycle management Effective • Rich metrics Identity & Data Network Threat Security access protection security protection management management

Encryption Azure VNET, VPN, NSG Azure Security Center (Disks, Storage, SQL)

Multi-Factor Application Gateway Microsoft Antimalware Azure Key Vault Azure Log Analytics Authentication (WAF), Azure Firewall for Azure

Role Based Access DDoS Protection Control Standard

Azure Active Directory (Identity Protection) ExpressRoute

+ Partner Solutions Partner-based Solutions

Sensor data fusion tools

Data Data Annotation Data extraction Extraction Preparation Labeling and annotation Microsoft Services

Data analysis

Blob Cognitive Services Services Logic Apps Functions HDInsight Search Databricks Cosmos DB Batch Data anonymization Data selection and transformation, PII data Workflow orchestration and redaction, annotation and training data managed services preparation Analyze your Data in one place

Big Data Use Cases Ingest & ETL Streaming Analytics & Machine Learning Data Aggregation Presentation

Functions Monitor Event Hubs App Insights Log Analytics Stream Machine Search Power BI Analytics Data Lake Learning Data Warehouse Analytics IoT Hub Data Factory CDN Batch Azure HDInsight

Blob Storage Pillars Open & Manageable & Scalable & Secure & Durable & Interoperable Cost Efficient Performant Compliant Available Manageable & Scalable & Secure & Cost Efficient Performant Compliant Deploy AI models in Azure for auto- labeling images to optimize for speed of labeling and possibly cost

Semantic Segmentation Each Pixel of the image is assigned a category

Object Detection & Classification Bounding box drawn around each object of interest

3D Point Cloud Labeling Objects of interest as assigned a category in 3D LIDAR point cloud Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)

Replay Tag Performance Simulate Hardware Simulation in the loop

Render/Convert Test-Drive

DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Partnership Opportunities

Open loop testing (e.g. ADTF)

Simulation tools for sensor and algorithm Test Scene Reconstruction Algorithm scoring Results Analysis validation repository and Replay

Microsoft Services Workflow Management Services

Comprehensive Validation framework Cosmos Blob Cycle Cloud CPU Logic Apps Batch Key Vault Service Bus PowerBI DB Compute

Verification and validation of training algorithms and sensors via open loop testing

Build and deploy deep learning models

Azure databricks Caffe2

Keras

MS cognitive toolkit Azure Scale out clusters databricks

TensorFlow

Notebooks Batch AI Azure ML Services

Scale out clusters

Streamline Scale compute Quickly evaluate AI development efforts resources in any environment and identify the right model

Leverage popular deep learning toolkits Choose VMs for your modeling needs Run experiments in parallel Develop your language of choice Process video using GPU-based VMs Provision resources automatically

Leverage your favorite deep learning frameworks

TensorFlow MS Cognitive Toolkit PyTorch Scikit-Learn ONNX Caffe2 MXNet Chainer Operationalize and manage models with ease

Azure Azure databricks ML services AKS ACI

Containers

Model MGMT, experimentation, Train and evaluate models and run history IoT edge

Bring models Proactively manage Deploy models to life quickly model performance closer to your data

Build and deploy models in minutes Identify and promote your best models Deploy models anywhere Iterate quickly on serverless infrastructure Capture model telemetry Scale out to containers Easily change environments Retrain models with Infuse intelligence into the IoT edge Simulation is the fastest way to “drive” the billions of miles needed to discover the breadth of real-world anomalies required to develop AD solutions. Sensor Simulation Scenario Simulation Engineering Simulation

Photo-realistic rendering of modeled sensors Scenario definition and simulation for path Mathematical based simulation for (Lidar, radar, RGB cameras) for the training planning and navigational maneuvers Structural CAD design, Fluid dynamics, and validation of Perception AI algorithms. (overtake, parallel parking, etc) using post Thermo dynamics, material properties perception object level simulation. Render once, test many times Design sensor properties and placement Run 7200 simulated scenarios covering ~5000 on vehicle options at scale before miles per day/per node hardware prototyping is needed. Simulate accidents and AD navigation to a safe stop Simulation/Reinforcement Learning Centric Model

Simulation Render/convert Engine

Test Vehicle Hardware Fleet Control Logic in the loop Sensor Fusion, Validation Process, Sample, Ingest/Store Training Algorithm Generate Reduce, Tag, QA Train Dataset Validation Engine Build Integrate Code

Software Production Performance in the loop Vehicle Fleet Simulation

F(x)

Test-Drive

“Drive Millions, Train on Billions” Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)

Replay Tag Performance Simulate Hardware Simulation in the loop

Render/Convert Test-Drive

DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Control Logic Software Validation in the loop Partner-based Solutions Generate Code F(x) Comprehensive test management framework Performance Hardware Simulation in the loop HIL solutions

Test-Drive System validation tools

Workflow management services Microsoft Services Managed services

Blob storage Batch GPU VM Active Directory Container service Embedded system validation via hardware-in- loop and software-in-loop © Copyright Microsoft Corporation. All rights reserved.