Microsoft Azure in Autonomous Driving Microsoft’s approach to automotive We complement OEMs and suppliers – not compete We ensure your data is always under your control We guarantee that the brand and customer experience belongs to you OEM System Engineering Process Engineering Development Product Validation Manufacturing & Service Feasibility Operations Regional Changes Retirement/ Study/concept and Architecture (s) and Upgrades replacement Exploration Maintenance System Validation Plan Lifecycle Processes Concepts of System Operations Validation System Verification Plan (System Acceptance) System System Verification & Requirements Deployment Subsystem Verification Plan (Subsystem Acceptance) High-Level Subsystem Design Verification Unit/Device Test Plan Unit/Device Detailed Design Testing Document/Approval Software/Hardware Development Field Installation Implementation Development Processes Time Line Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Replay Tag Performance Simulate Hardware Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE • Cloud and Analytics partner for • Open source AD Platform • OpenADx ACM, an autonomous and smart • 200+ Member Consortium • Interoperable Eclipse Framework mobility test facility • Microsoft is the cloud provider for • Leveraged by all OEMs, tier ones and Project Apollo worldwide with the technology start ups exception of China • Engaged with ACM to influence standards Industry Academia Government LG’s autonomous vehicle program had unique requirements, including portability, security, and fast turnaround time; Data Box Disk was the perfect solution. “We needed a way to transfer massive amounts of data for our autonomous vehicle projects, based all around the world. The solution needed to be portable, simple to use, cost-effective and, of course, very secure. The Azure Data Box Disk met all of those criteria.” – Hyoyuel (Andy) Kim, Senior Manager, LG Electronics. Products and services Organization size Industry Country Azure Data Box Disk 91,000 Automotive Worldwide Azure Blob Storage electronics Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Replay Tag Performance Simulate Hardware Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Azure Express Route Azure Compute Databricks & HDInsight Azure IoT Edge Azure Storage, Data Lake Gen2 Azure Batch AI Azure Machine Learning Container and Kubernetes , Visual Studio Team Services Azure Data Box Cycle Cloud Cognitive Services Azure Key Vault, AAD SOFTWARE TOOLS AND DEVOPS PLATFORM CI/CD flows Machine Learning Azure Container Source Container Registry Control Service DATA INGEST Data Factory Virtual machine Cycle Cloud HDInsight, Azure GPU Azure VM Cosmos DB In-car Databricks Logic App data Async storage upload Kubernetes Azure blob Service Simplygon storage (cool, ML PowerBI archive) CosmosDB Data Fast Advanced HDInsight reduction, upload Stream Analytics Batch Analytics Databricks Service Bus redaction IoT Edge ADLS Gen 2 and Azure blob storage Hot Tier Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Replay Tag Performance Simulate Hardware Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Offline Data Transfer Online Data Transfer Data Box Data Box Disk Data Box Heavy Data Box Gateway Data Box Edge • Capacity: 100 TB • Capacity: 8TB ea.; 40TB/order • Capacity: 1000TB • Virtual device provisioned in • Local Cache Capacity: ~12 TB • Weight: ~50 lbs • Secure, ruggedized USB • Weight 500+ lbs your hypervisor • Includes Data Box Gateway • Secure, ruggedized drives orderable in packs of • Secure, ruggedized • Supports storage gateway, and Azure IoT Edge. appliance 5 (up to 40TB). appliance SMB, NFS, Azure blob, files • Currently in Preview • Currently in Preview • Currently in preview • Currently in preview • Data Box enables bulk migration to Azure when • Perfect for projects that • Same service as Data Box, • Virtual network transfer • Data Box Edge manages network isn’t an option. require a smaller form factor, but targeted to petabyte- appliance (VM), runs on your uploads to Azure and can e.g., autonomous vehicles. sized datasets. choice of hardware. pre-process data prior to upload. Network Data Transfer Edge Compute Order Send Fill Return Upload Cloud to Edge Edge to Cloud Pre-processing ML Inferencing Massive Data Ingest – ExpressRoute Ingest up to PB of data daily with Azure Networking Express Route Service Private, high speed network connections Connecting your data center and Azure Predictable performance and Secure 100 + Express Route partners Data Storage – Azure Blob Storage Store up to Exabytes of data in massively scalable object storage for unstructured data • Foundational service for Microsoft Scalable • >40 million transactions per second • Multi-PB accounts • 50+Gbps account throughput Performant • Continued enhancements under Frequently Less frequently Rarely development accessed data accessed data accessed data Secure & • Client & Service Encryption • AAD Integration + ACLs Lower capacity Lowest transaction Lowest capacity Compliant • Broad & deep compliance portfolio cost cost cost • Multiple redundancy options Immediate (ms) Immediate (ms) Hours Durable • Strong consistency, data integrity • Policy: Versioning & WORM locks Cost • Integrated storage tiers • Lifecycle management Effective • Rich metrics Identity & Data Network Threat Security access protection security protection management management Encryption Azure Active Directory VNET, VPN, NSG Azure Security Center (Disks, Storage, SQL) Multi-Factor Application Gateway Microsoft Antimalware Azure Key Vault Azure Log Analytics Authentication (WAF), Azure Firewall for Azure Role Based Access DDoS Protection Control Standard Azure Active Directory (Identity Protection) ExpressRoute + Partner Solutions Partner-based Solutions Sensor data fusion tools Data Data Annotation Data extraction Extraction Preparation Labeling and annotation Microsoft Services Data analysis Blob Cognitive Services Services Logic Apps Functions HDInsight Search Databricks Cosmos DB Batch Data anonymization Data selection and transformation, PII data Workflow orchestration and redaction, annotation and training data managed services preparation Analyze your Data in one place Big Data Use Cases Ingest & ETL Streaming Analytics & Machine Learning Data Aggregation Presentation Functions Monitor Event Hubs App Insights Log Analytics Stream Machine Search Power BI Analytics Data Lake Learning Data Warehouse Analytics IoT Hub Data Factory CDN Batch Azure HDInsight Blob Storage Pillars Open & Manageable & Scalable & Secure & Durable & Interoperable Cost Efficient Performant Compliant Available Manageable & Scalable & Secure & Cost Efficient Performant Compliant Deploy AI models in Azure for auto- labeling images to optimize for speed of labeling and possibly cost Semantic Segmentation Each Pixel of the image is assigned a category Object Detection & Classification Bounding box drawn around each object of interest 3D Point Cloud Labeling Objects of interest as assigned a category in 3D LIDAR point cloud Process, Sensor/ Algorithm Train Control Logic Software Sample, Reduce Testing (Open Loop) Validation in the loop Generate Test Vehicle Ingest/ Store Integrate Build Code F(x) Replay Tag Performance Simulate Hardware Simulation in the loop Render/Convert Test-Drive DATA INGEST & CURATE TEST | TRAIN | SIMULATE BUILD | VALIDATE Partnership Opportunities Open loop testing (e.g. ADTF) Simulation tools for sensor and algorithm Test Scene Reconstruction Algorithm scoring Results Analysis validation repository and Replay Microsoft Services Workflow Management Services Comprehensive Validation framework Cosmos Blob Cycle Cloud CPU Logic Apps Batch Key Vault Service Bus PowerBI DB Compute Verification and validation of training algorithms and sensors via open loop testing Build and deploy deep learning models Azure databricks Caffe2 Keras MS cognitive toolkit Azure Scale out clusters databricks TensorFlow Notebooks Batch AI Azure ML Services Scale out clusters Streamline Scale compute Quickly evaluate AI development efforts resources in any environment and identify the right model Leverage popular deep learning toolkits Choose VMs for your modeling needs Run experiments in parallel Develop your language of choice Process video using GPU-based VMs Provision resources automatically Leverage your favorite deep learning frameworks TensorFlow MS Cognitive Toolkit PyTorch Scikit-Learn ONNX Caffe2 MXNet Chainer Operationalize and manage models with ease Azure Azure databricks ML services AKS ACI Containers Model MGMT, experimentation, Train and evaluate models and run history IoT edge Bring models Proactively manage Deploy models to life quickly model performance closer to your data Build and deploy models in minutes Identify and promote your best models Deploy models anywhere Iterate quickly on serverless infrastructure Capture model telemetry Scale out to containers Easily change environments Retrain models with APIs Infuse intelligence into the IoT edge Simulation