Qualcomm® Cloud AI 100 Announcement
Under Embargo until Sept 16 at 6:30am PT Qualcomm Cloud AI is a product of Qualcomm Technologies, Inc. and/or its subsidiaries. Qualcomm Cloud AI 100 addressing edge-to-cloud industries
Data Center/ 5G Cloud Edge Infrastructure 5G Edge Box
2 Datacenter Edge Box z 5G Infra
Pedestrian alert Crossing & blind spot assist z
Personalized Purchase recommendations Road safety Intersection management assist
Personalized Purchase advertisements
Reinventing the Powering the shopping Pioneering communication of the future safety © 2020 Qualcomm experience 3 A reminder why we are doing this… COMPUTING IS MOVING TOWARD THE EDGE; SIGNIFICANT OPPORTUNITY FOR ENTRY
Explosive growth in Inference; High and Medium performance addressed well by Qualcomm Cloud AI 100
d Source: 2020 Omdia 4 AI Performance 15+ Continuum TOPS Snapdragon 865+
Snapdragon 3rd Gen Qualcomm® Snapdragon™ Snapdragon Snapdragon 820A Automotive Cockpit Platforms 845 855
Snapdragon 8cx 7-15 Snapdragon Snapdragon TOPS 730/G 765 Snapdragon 7c 5-10 Snapdragon TOPS 665
Snapdragon Snapdragon Snapdragon XR1/2 630/36 670 0.5-1+ Snapdragon Snapdragon Snapdragon Snapdragon 3-5 427 429 439 450 TOPS TOPS
Qualcomm® Qualcomm® Qualcomm® QCS400 QCS603 QCS605
5 Qualcomm Snapdragon, Qualcomm QCS400, Qualcomm QCS603, and Qualcomm QCS605 are products of Qualcomm Technologies, Inc. and/or its subsidiaries. Qualcomm Full Production Cycle Cloud AI 100 Final Silicon hitting peak TOPS and Performance per watt 400 TOPS PCIe Qualcomm Cloud AI 100 200 TOPS Qualcomm DM.2 Cloud AI 100 >50 SD710SD 3 rd Gen Qualcomm® Snapdragon SD730GSD670SD730SD820AutomotiveXR1 Cockpit Platforms SD427630/36SD429SD845SD450820ASD855SD439 TOPS QCSQCS 0.5-1+ SD660SD835SD665 >15 QCS603QCS400 QCS60580098053 TOPS TOPS DM.2e
6 An architecture shift in AI cloud inferencing
Purpose Built AI Accelerators
FPGA >10X Improvement >10X Improvement
7 Industry Leading Inference Performance Qualcomm Cloud AI 100 outperforms Competition by 4~10x (Inference/Sec/W)
ResNet-50 Performance vs. Competition Intel Cascade Lake CPU 1x 440W
Nvidia T4 GPU 10x 70W
Intel/Habana Goya ASIC 26x 100W
Qualcomm Cloud AI 100 20W 106x
8 Nvidia website: https://developer.nvidia.com/deep-learning-performance-training-inference Habana: Goya Inference Platform White Paper Intel: measured on AWS Performance & Efficiency Leadership ResNet-50 Benchmark
30000
>50 Watt Cloud AI 100 PCIe <50 Watt 25000 A100
20000 TSP
15000 Goya Cloud AI 100 DM.2
10000
Throughput (Inferences/Sec) Throughput 5000 Cloud AI 100 DM.2e V100 T4 0
350 300 250 200 150 100 50 0 Power (Watt)
Power (Watt) - Lower is Better
Source: Habana and Nvidia websites, Linley report for . Batch size = 8 for Nvidia T4, V100, Habana Goya, Groq TSP and Cloud AI100. Batch size unknown for Nvidia A100 9 Hardware Architecture
• Up to 400 TOPS • Power • DM.2e @ 15W • DM.2 at 25W • PCIe/HHHL @ 75W • AI Core (AIC) - Up to 16 cores • Precision – INT8, INT16, FP16, FP32 • On-die SRAM – Up to 144 MB • 4x64 LPDDR4x (2.1GHz) with inline ECC • Up to 32GB on card DRAM • PCIe Gen 3/4 - Up to 8 lanes
10 Computer Language Recommendation Speech Autonomous Applications Vision Translation System
Other ResNet DIN Models SSD GNMT BERT DLRM Models
Frameworks
Exchange Formats
Runtimes
AIC Apps SDK AIC Platform SDK SDKs (Compiler, simulator, sample codes) (Runtime, APIs, Kernel Drivers, tools)
AI Inference Accelerator Cards Hardware
11 Introducing… The Qualcomm Cloud AI 100 Edge Development Kit
12 13 Qualcomm Cloud AI 100 Summary
Full Production Cycle/Final Silicon • Sampling now to multiple customers • Shipping 1st Half 2021
Edge Development Kit • Sampling October 2020 • Single Solution: one-stop-shop with AI inferencing, host processor and 5G connectivity • Pre-certified modem module with industry leading 5G • Cutting-edge performance per Watt
Source sample text 14 Product Brief Qualcomm® Cloud AI 100 Qualcomm® Cloud AI 100 Edge Development Kit
Schedule Sampling Now Schedule Shipping October 2020 Commercial launch 1H 21 Targeted Application 5G Intelligent Edge Compute Process Node 7nm Appliance Dimensions (mm) 231 (w) x 250 (h) x 84 (d) without Card Types Dual M.2 (edge): 15W TDP antenna and stand Dual M.2: 25W TDP 231 (w) x 504 (h) x 120 (d) with PCIe (HHHL): 75W TDP antenna and stand Card Performance (Raw TOPS) Dual M.2 (edge): 70 TOPS Operating Temp Range (Ambient) 0C – 50C Dual M.2: 200 TOPS PCIe: 400 TOPS Host SOC Qualcomm® Snapdragon™ 865 AI Cores Up to 16 cores Mobile Platform Qualcomm® Kryo™ 585 CPU cores up to 2.84GHz On Die SRAM 144MB (9MB Each AI Core) Integrated video pipeline to process 24 streams of FHD @ 30fps On Card DRAM Up to 32GB w/ 4x64 LPDDR4x @ simultaneously 2.1GHz 12GB LPDDR5 memory Operating System CentOS 8.0 PCIe (connection to the host) 8 lane Gen3/4 (PCIe) or 4 lane AI Accelerator Qualcomm Cloud AI 100 (DM.2 Gen3/4 (Dual M.2) edge variant) Data Types INT8, INT16, FP16, FP32 Connectivity Snapdragon X55 5G modem on M.2 module (global sub6 variant)
Ethernet (LAN and WAN) 15 Qualcomm AI Cloud, Qualcomm Snapdragon and Qualcomm Kryo is a product of Qualcomm Technologies, Inc. and/or its subsidiaries. Storage NVMe SSD Thank you
Follow us on: For more information, visit us at: www.qualcomm.com & www.qualcomm.com/blog
Nothing in these materials is an offer to sell any of the References in this presentation to “Qualcomm” may mean Qualcomm components or devices referenced herein. Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries or business units within the Qualcomm corporate structure, as applicable. ©2018-2020 Qualcomm Technologies, Inc. and/or its Qualcomm Incorporated includes Qualcomm’s licensing business, QTL, affiliated companies. All Rights Reserved. and the vast majority of its patent portfolio. Qualcomm Technologies, Qualcomm is a trademark of Qualcomm Incorporated, Inc., a wholly-owned subsidiary of Qualcomm Incorporated, operates, registered in the United States and other countries. along with its subsidiaries, substantially all of Qualcomm’s engineering, Other products and brand names may be trademarks or research and development functions, and substantially all of its product registered trademarks of their respective owners. and services businesses, including its semiconductor business, QCT.
Source sample text Confidential – Qualcomm Technologies, Inc. and/or its affiliated companies – May Contain Trade Secrets