Microsoft Azure in Autonomous Driving Microsoft’s approach to automotive
We complement OEMs and suppliers – not compete
We ensure your data is always under your control
We guarantee that the brand and customer experience belongs to you
OEM System Engineering Process
Engineering Development Product Validation Manufacturing & Service
Feasibility Operations Regional Changes Retirement/ Study/concept and Architecture (s) and Upgrades replacement Exploration Maintenance System Validation Plan Lifecycle Processes Concepts of System Operations Validation
System Verification Plan (System Acceptance) System System Verification & Requirements Deployment Subsystem Verification Plan (Subsystem Acceptance) High-Level Subsystem Design Verification Unit/Device Test Plan Unit/Device Detailed Design Testing Document/Approval
Software/Hardware Development Field Installation
Implementation Development Processes Time Line Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)
Replay Tag Performance Simulate Hardware Simulation in the loop
Render/Convert Test-Drive
DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE • Cloud and Analytics partner for • Open source AD Platform • OpenADx ACM, an autonomous and smart • 200+ Member Consortium • Interoperable Eclipse Framework mobility test facility • Microsoft is the cloud provider for • Leveraged by all OEMs, tier ones and Project Apollo worldwide with the technology start ups exception of China • Engaged with ACM to influence standards
Industry
Academia Government
LG’s autonomous vehicle program had unique requirements, including portability, security, and fast turnaround time; Data Box Disk was the perfect solution.
“We needed a way to transfer massive amounts of data for our autonomous vehicle projects, based all around the world. The solution needed to be portable, simple to use, cost-effective and, of course, very secure. The Azure Data Box Disk met all of those criteria.” – Hyoyuel (Andy) Kim, Senior Manager, LG Electronics.
Products and services Organization size Industry Country Azure Data Box Disk 91,000 Automotive Worldwide Azure Blob Storage electronics Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)
Replay Tag Performance Simulate Hardware Simulation in the loop
Render/Convert Test-Drive
DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE
Azure Express Route Azure Compute Databricks & HDInsight Azure IoT Edge
Azure Storage, Data Lake Gen2 Azure Batch AI Azure Machine Learning Container and Kubernetes , Visual Studio Team Services Azure Data Box Cycle Cloud Cognitive Services Azure Key Vault, AAD SOFTWARE TOOLS AND DEVOPS PLATFORM
CI/CD flows Machine Learning Azure Container Source Container Registry Control Service
DATA INGEST
Data Factory
Virtual machine
Cycle Cloud HDInsight, Azure GPU Azure VM Cosmos DB In-car Databricks Logic App data Async storage upload Kubernetes Azure blob Service Simplygon storage (cool, ML PowerBI archive) CosmosDB Data Fast Advanced HDInsight reduction, upload Stream Analytics Batch Analytics Databricks Service Bus redaction IoT Edge ADLS Gen 2 and Azure blob storage Hot Tier Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)
Replay Tag Performance Simulate Hardware Simulation in the loop
Render/Convert Test-Drive
DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Offline Data Transfer Online Data Transfer
Data Box Data Box Disk Data Box Heavy Data Box Gateway Data Box Edge • Capacity: 100 TB • Capacity: 8TB ea.; 40TB/order • Capacity: 1000TB • Virtual device provisioned in • Local Cache Capacity: ~12 TB • Weight: ~50 lbs • Secure, ruggedized USB • Weight 500+ lbs your hypervisor • Includes Data Box Gateway • Secure, ruggedized drives orderable in packs of • Secure, ruggedized • Supports storage gateway, and Azure IoT Edge. appliance 5 (up to 40TB). appliance SMB, NFS, Azure blob, files • Currently in Preview • Currently in Preview • Currently in preview • Currently in preview • Data Box enables bulk migration to Azure when • Perfect for projects that • Same service as Data Box, • Virtual network transfer • Data Box Edge manages network isn’t an option. require a smaller form factor, but targeted to petabyte- appliance (VM), runs on your uploads to Azure and can e.g., autonomous vehicles. sized datasets. choice of hardware. pre-process data prior to upload.
Network Data Transfer Edge Compute
Order Send Fill Return Upload Cloud to Edge Edge to Cloud Pre-processing ML Inferencing Massive Data Ingest – ExpressRoute Ingest up to PB of data daily with Azure Networking Express Route Service
Private, high speed network connections
Connecting your data center and Azure
Predictable performance and Secure
100 + Express Route partners
Data Storage – Azure Blob Storage Store up to Exabytes of data in massively scalable object storage for unstructured data
• Foundational service for Microsoft Scalable • >40 million transactions per second • Multi-PB accounts
• 50+Gbps account throughput Performant • Continued enhancements under Frequently Less frequently Rarely development accessed data accessed data accessed data Secure & • Client & Service Encryption • AAD Integration + ACLs Lower capacity Lowest transaction Lowest capacity Compliant • Broad & deep compliance portfolio cost cost cost • Multiple redundancy options Immediate (ms) Immediate (ms) Hours Durable • Strong consistency, data integrity • Policy: Versioning & WORM locks
Cost • Integrated storage tiers • Lifecycle management Effective • Rich metrics Identity & Data Network Threat Security access protection security protection management management
Encryption Azure Active Directory VNET, VPN, NSG Azure Security Center (Disks, Storage, SQL)
Multi-Factor Application Gateway Microsoft Antimalware Azure Key Vault Azure Log Analytics Authentication (WAF), Azure Firewall for Azure
Role Based Access DDoS Protection Control Standard
Azure Active Directory (Identity Protection) ExpressRoute
+ Partner Solutions Partner-based Solutions
Sensor data fusion tools
Data Data Annotation Data extraction Extraction Preparation Labeling and annotation Microsoft Services
Data analysis
Blob Cognitive Services Services Logic Apps Functions HDInsight Search Databricks Cosmos DB Batch Data anonymization Data selection and transformation, PII data Workflow orchestration and redaction, annotation and training data managed services preparation Analyze your Data in one place
Big Data Use Cases Ingest & ETL Streaming Analytics & Machine Learning Data Aggregation Presentation
Functions Monitor Event Hubs App Insights Log Analytics Stream Machine Search Power BI Analytics Data Lake Learning Data Warehouse Analytics IoT Hub Data Factory CDN Batch Azure HDInsight
Blob Storage Pillars Open & Manageable & Scalable & Secure & Durable & Interoperable Cost Efficient Performant Compliant Available Manageable & Scalable & Secure & Cost Efficient Performant Compliant Deploy AI models in Azure for auto- labeling images to optimize for speed of labeling and possibly cost
Semantic Segmentation Each Pixel of the image is assigned a category
Object Detection & Classification Bounding box drawn around each object of interest
3D Point Cloud Labeling Objects of interest as assigned a category in 3D LIDAR point cloud Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)
Replay Tag Performance Simulate Hardware Simulation in the loop
Render/Convert Test-Drive
DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Partnership Opportunities
Open loop testing (e.g. ADTF)
Simulation tools for sensor and algorithm Test Scene Reconstruction Algorithm scoring Results Analysis validation repository and Replay
Microsoft Services Workflow Management Services
Comprehensive Validation framework Cosmos Blob Cycle Cloud CPU Logic Apps Batch Key Vault Service Bus PowerBI DB Compute
Verification and validation of training algorithms and sensors via open loop testing
Build and deploy deep learning models
Azure databricks Caffe2
Keras
MS cognitive toolkit Azure Scale out clusters databricks
TensorFlow
Notebooks Batch AI Azure ML Services
Scale out clusters
Streamline Scale compute Quickly evaluate AI development efforts resources in any environment and identify the right model
Leverage popular deep learning toolkits Choose VMs for your modeling needs Run experiments in parallel Develop your language of choice Process video using GPU-based VMs Provision resources automatically
Leverage your favorite deep learning frameworks
TensorFlow MS Cognitive Toolkit PyTorch Scikit-Learn ONNX Caffe2 MXNet Chainer Operationalize and manage models with ease
Azure Azure databricks ML services AKS ACI
Containers
Model MGMT, experimentation, Train and evaluate models and run history IoT edge
Bring models Proactively manage Deploy models to life quickly model performance closer to your data
Build and deploy models in minutes Identify and promote your best models Deploy models anywhere Iterate quickly on serverless infrastructure Capture model telemetry Scale out to containers Easily change environments Retrain models with APIs Infuse intelligence into the IoT edge Simulation is the fastest way to “drive” the billions of miles needed to discover the breadth of real-world anomalies required to develop AD solutions. Sensor Simulation Scenario Simulation Engineering Simulation
Photo-realistic rendering of modeled sensors Scenario definition and simulation for path Mathematical based simulation for (Lidar, radar, RGB cameras) for the training planning and navigational maneuvers Structural CAD design, Fluid dynamics, and validation of Perception AI algorithms. (overtake, parallel parking, etc) using post Thermo dynamics, material properties perception object level simulation. Render once, test many times Design sensor properties and placement Run 7200 simulated scenarios covering ~5000 on vehicle options at scale before miles per day/per node hardware prototyping is needed. Simulate accidents and AD navigation to a safe stop Simulation/Reinforcement Learning Centric Model
Simulation Render/convert Engine
Test Vehicle Hardware Fleet Control Logic in the loop Sensor Fusion, Validation Process, Sample, Ingest/Store Training Algorithm Generate Reduce, Tag, QA Train Dataset Validation Engine Build Integrate Code
Software Production Performance in the loop Vehicle Fleet Simulation
F(x)
Test-Drive
“Drive Millions, Train on Billions” Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x)
Replay Tag Performance Simulate Hardware Simulation in the loop
Render/Convert Test-Drive
DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Control Logic Software Validation in the loop Partner-based Solutions Generate Code F(x) Comprehensive test management framework Performance Hardware Simulation in the loop HIL solutions
Test-Drive System validation tools
Workflow management services Microsoft Services Managed services
Blob storage Batch GPU VM Active Directory Container service Embedded system validation via hardware-in- loop and software-in-loop © Copyright Microsoft Corporation. All rights reserved.